Small Business Resources, Business Advice and Forms from AllBusiness.com

Business Exchange

Extreme risks: the financial services industry addresses ever tougher continuity standards.

By:Bielski, Lauren
Publication: ABA Banking Journal
Date:Saturday, March 1 2008
Subject: Standards
Company: Citibank N.A. (New York, New York)
Product: National commercial banks, State commercial banks, Commercial banks, not elsewhere classified
Location: Canada

[ILLUSTRATION OMITTED]

Bungee jumping, skydiving, or other extreme sports may give an adrenaline rush for many a youthful, plucky soul, but as for the danger and the risk, participants choose it. Something that could be called "extreme context" seems to have become a base element of doing business these days, but you don't choose it and you can't opt out.

Events can arrive whether you operate in, or near, a flood zone, hurricane alley, or high-terror-alert corridors. Preparation counts whether you conduct banking on North Tryon Street, Charlotte, or Park Avenue, New York, or Main Street in the seemingly quiet hamlet of Anywhere, USA.

Given the range of possible disrupting scenarios, Citibank takes an "all hazards" approach to continuity planning, according to Greg Gist, senior policy advisor and head of Citi's Pandemic Task Force and office of Business Continuity. Loosely translated, the approach refers to a risk-based planning method that incorporates emergency response guidelines for moving people quickly in the case of a natural disaster or moving to a back-up office location, among other factors.

The global bank's continuity effort touches on actual exposures the bank faces per line-of-business input, and takes in all aspects of mission-critical operations. As part of this, Citibank examines how key processes can be backed up or reengineered for resilience.

"As a global organization we face any number of potential hazards from earthquakes, flood, and fire. to IT-related strains on our system." Gist says. "We analyze processes to limit the scope of possible damage. In other words, you manage to control the effect, since you can't control what you might be hit with."

While Gist concurs that no one plan can work for all banks, he does think there is a general best practice approach to making a plan worth having. In brief, it might be described as derived from a deep analysis of what the regulators want and what your organization needs given its risk tolerance, it's also, to repeat, comprehensive--covering people, processes, technology, and facilities ... the works. Some in the industry think of business continuity (BC) as doing what's necessary to set up a "shadow" organization that will take you from incident response through various phases of recovery.

Thinking under pressure

Sure, this is costly planning for something that may' never happen, but, in the words of one executive, "It's better to have a somewhat costly plan for an emergency that isn't used than to have to explain post-catastrophe why you weren't prepared."

And, it's better to have tested the plan when the stakes aren't as high as they could be.

Michael Nolan, chairman and president, $371 assets Fifth District Savings Bank, New Orleans agrees. In his case he was hit with a whopper of a weather event, one called Katrina, which forced him and other key staffers to improvise under pressure. It was something the bank did well with, but hopes to never live through again.

"We were lucky to find a partner in St. Martin Bank and Trust, because with 70% of the entire city under 4 to 12 feet of water, our previous plan was rendered unusable," says Nolan. instead, key team members operated in a conference room out of a St. Martin's office 120 miles from downtown New Orleans in Cajun country.

A broader group of personnel acquired emergency housing--at friends' homes, local B&Bs, and inns--and took part in a recovery plan that unfolded over several weeks. "That became the basis of our current emergency response plan," Nolan recalls.

Months later, as a part of a continuity update, the bank redesigned its IT environment so that, in effect, the downtown office, which happened to be one of the oldest buildings and one that was the hardest to weather-protect, wasn't the primary feed for its five branches. "Now the core processor and other technologies are directly linked to the branches independently," says Nolan. "We also have a new vendor providing our backup facility."

Plan, do, act...

To say that government and private industry execs were shaken awake to the need for--and rigors of--industrial grade business continuity planning by terrorism, weather events, power outages, and random disruptions over the last eight years is possibly the ultimate understatement.

One thing is true, if anyone ever thought that having a paper-based plan designed for "auditors eyes only" was cutting it, events like Katrina dramatically showed otherwise. The cycle of continuity is exactly that, a cycle, where "plan, do, act, check--and refine" should be part of plan development and maintenance, notes John DiMaria, product manager for business continuity, BSI Management Systems America, Reston, Va.

"Those that work on the testing and refining the testing aspects of the cycle are head and shoulders above their less-tested peers," says DiMaria.

As an avid plan tester, Citibank uses collaborative content management-style systems to store the digital volumes of its continuity-related plans (e.g., system recovery guidelines and similar documentation). The continuity team works closely with line-of-business executives and risk managers to assess actual risk footprints, rather than relying on "guestimates" of operational procedures, notes John Odermatt, global head of Citi's Office of Continuity. "Yet I also believe the industry--not just this bank or other large banks--has learned lessons," says Odermatt, "I think the industry shines, in particular, when it comes to systems testing and recovery."

In its efforts, Citi is admittedly a best practice performer, one whose techniques can't easily be emulated by smaller banks with lesser budgets, says Lee Milligan, senior project manager with Strohl Systems, King of Prussia, Pa. Still, Milligan works with many small banks and credit unions and says that engaging in an analysis of key business operations to make, for example, better use of core vendor outsource programs related to BC is something that nearly everyone can afford.

Need to test

In lockstep with industry improvements since 2001, however, come higher expectations. President Bush in August signed "Implementing the 9/11 Committee Recommendations Act," or HR 1 (PL 110-53), into law, asking as part of that, for voluntary certifications of continuity-related fitness using National Fire Protection Association (NFPA) 1600 standards as a guideline. Testing is a part of demonstrating fitness.

In September, the Financial Services Sector Coordinating Council (FSSCC), in effect the financial services industry arm of the Department of Homeland Security, led a vast, several-week-long virtual desktop exercise in which thousands of banks participated as part of the government's pandemic preparation.

Overall, the exercise was characterized as a solid foundation effort. "It was a good way to see what we knew and what we didn't--a necessary first step in hammering out an effective response," notes Doug Johnson, a senior policy analyst with the ABA who has been heavily involved in continuity programs. "It will help banks get a better sense of how to refine individual plans," he says.

As of this writing, the Federal Financial Institutions Examination Council (FFIEC) is set to rollout new guidance on testing just a few years after having updated the Business Continuity Information Technology Examination Handbook in 2003.

"This time, FFIEC didn't say, 'We'll agree about what to test.' They said, 'The following elements of your plan will be tested and you will be able to demonstrate that tests occurred and what the outcomes were," relates Strohl's Milligan, who is familiar with FFIEC's revisions. "They made system recovery and emergency response testing a bigger priority."

Certainly, there are banks, including Citibank, Wachovia, and CIBC, that reflect the discipline's perspective depth, running tests that span entire lines of business and support departments such as Finance or HR, notes Graeme Jannaway, managing director Jannaway & Associates, Toronto. "Many more banks routinely run IT-only recovery exercises."

He went on to describe a program that CIBC runs, which he particularly admires. "Entire divisions are told, as employees show up at their office in the morning, 'head over to the bus parked around the bend. You're working at the backup facility this week'," he says.

Some banks are even beginning to grill suppliers over their disaster recovery and continuity efforts to make sure that third parties won't introduce faults into the system.

Still, institutions smaller than the top 100 generally have only scratched the surface of deep plan testing, said one consultant who asked not to be named.

It's not just a banking issue. Philip Rothstein, publisher of the Rothstein Catalog on Disaster Recovery, and president of Rothstein Associates, Brookfield, Conn., isn't wowed by the performance of any industry. "It's a bit disturbing that the only book out there on continuity testing seems to be mine, which was written 14 years ago. It's just one indicator, but it does make me question how seriously the testing phrase is being taken," says the BC consultant. Obviously, competing business priorities can take attention, and so can a culture of being careful.

Because more than one expert admitted on background that program refinement and testing--"testing to plan and not planning to test," in the words of one executive--means that the industry needs to be more willing to see where its programs really are, even though testing is expensive and some lessons learned during tests can be embarrassing.

The fact is, progress not perfection should always be the aim as the inherent complexity of the industry makes it tough to craft resiliency around, notes Steve Elliot, Elliot Consulting Services, Tampa. He points out that growth through M&A is just one more top-of-mind reason why today's carefully crafted continuity plan can become out-of-date in a flash.

"It's important to be set up to modify the plan frequently," says Elliot. "That means making use of some sort of content management system or portal-based technology so that information can be centrally collected, vetted, and posted for all appropriate constituents." Elliot also advises banks to periodically update all employees on shifts in approach that may touch on their duties, and adds that all employees should be trained in appropriate emergency response roles.

"Just because the emergency response and recovery team may be a much smaller crew of key executives," says Elliot, "doesn't mean that everyone in the company doesn't have a role to play in going from incidence response to the phases of recovery.

Research shows some prep done

Graeme Jannaway has decades-long experience in continuity, which has included stints as president and chairman of the board of the Disaster Recovery Institute Canada; and former director of DRI International.

He says recent events shaping the continuity discipline fall into two key narratives worth noting. The first is the greater input and oversight from the "c-suite" at top-tier banks. "They don't need to be sold on the importance of this."

The second regards standardization, which has codified and reshaped thinking on how to become resilient.

"NFPA 1600 is the grandaddy of standards driving U.S. legislation and thought," he says, adding that Sarbanes Oxley and Basel II also touch on continuity as part of broader efforts to promote risk assessment.

"In Canada, Z1600 as part of the code of Canadian Public Safety is shaping programs and in the UK BSI 59999, which specifies requirements for implementing, operating, improving, and documenting continuity plans, is the big driver."

And while this can be conducive to, and indicative of, a kind of compliance mindset, according to experts interviewed, it can also be a solid driver of program improvements.

As with most industries with a strong dependence on IT transactions and digital records, banks have clambered up a steep learning curve in the years since September 2001. "More financial services firms have attempted to refine plans to incorporate some of the industry learnings caused by broad regional events that disrupt primary offices," says Dr. Tom Phelan, a founding member of the U.S. Department of Homeland Security and president of Strategic Teaching Associates based in Liverpool, N.Y.

"On the whole banks have stepped up scrutiny, professionalism, and generally taken more action in areas like vulnerability assessment since 2004," he adds.

Banks generally get high marks on foundation efforts such as defining mission critical assets and conducting personnel assessments, risk analysis and general planning.

As for specific strengths, for starters, IT-savvy banks know their systems. The IT- and facilities-focused discipline of disaster recovery has gained heft--and leverage--from new recovery tools. These include file recovery programs, various diagnostic programs, techniques like virtualization, and inventive ways of building applications and system environments, experts say.

"Tech developments like services oriented architecture and a more standardized approach to building or enhancing IT environments, plus recovery lessons learned from first-to-the-field practitioners such as FX divisions of banks and capital markets firms have helped banks build on the natural strength they always had with systems," says Washington, D.C.-based Gil Brodnitz, executive partner with Accenture's Strategic IT Effectiveness Practice.

(Generally, ABABJ heard that Wall Street firms tend to move the quickest on recovery and backup-related IT spending, although at least one source for this article disagreed and said that bankers, more risk averse by nature, take IT asset protection more seriously.)

"Some types of IT assets, including databases and networks, are easier to test now by using virtualization," Brodnitz adds. (You can think of "virtualization" as working with a copy, a simulation of the live IT environment.)

Still, his San Francisco-based colleague, Gary Curtis, points out that even in IT, challenges remain. The global managing director for Accenture's Technology Consulting Practice, Curtis points out that the industry, from the most rigorous to the unschooled in the bunch, is chasing a moving target.

"What concerns me--what banks have to contend with--is the complexity of their transactions and information vulnerability. It is tough to return to the environment at the point of failure," says Curtis.

The new continuity status quo

According to recent research, many more companies in general are thinking harder, and in more detailed fashion, about BC. Cambridge, Mass.-based Forrester Research and Disaster Recovery Journal recently published a research report, The State of DR Preparedness, based on responses from 250 global continuity-related decision makers in a number of fields.

While not limited to banking, its numbers are reasonably illustrative for the industry, notes Jannaway, who is familiar with the survey.

The good news is that, generally, continuity is becoming a higher priority, although survey respondents were made up of sophisticated practitioners rather than a random sample, according to DRJ. The survey noted that 79% had formal plans (banks, of course, have long been mandated to have them). About a third indicated that they continuously test their plan, and 14% update it at least quarterly. Increasingly, companies prefer to make use of dedicated IT recovery infrastructure, and site separation is increasing.

And yet challenges remain as 45% of respondents spend less than $500,000 on disaster recovery planning, testing, and maintenance.

All admitted that testing and refining their plans was the most difficult aspect of program development. (And, true continuity testing would be even more expensive than the kinds of IT-only tests that the responses cited.)

The right people ...

As an industry, banks aren't backing off all that much on plan improvement despite a backsliding economy and other critical business challenges that are putting pressure on earnings, says Jannaway.

Many in banking also get credited for participation at a fairly senior level of the organization and putting some budget behind their efforts. The biggest banks have dedicated staffs that operate as a subdivision of operations and aren't IT-only in scope. Professionals also take their jobs seriously. "Bank industry continuity execs are actively involved members of organizations such as the Business Continuity Institute," notes Rothstein. "There's also more interest in individual certification--and in finding employees suited to continuity work by temperament and training," he explains. There are, he points out, numerous individual certifications that individuals can get including Disaster Recovery Institute International's Certified Business Continuity Professional certification (see page 32).

However, the command team of incidence response needs to involve the right people. "An incident command team--the sort of team that the government now mandates for all federal, state, and local agencies--needs to be headed by perhaps a chief operating officer, someone who isn't the president of the company," says Tom Frangione, president and CEO, Simply Continuous, a new continuity consultancy in San Francisco that works with customers in planning and testing.

"The president needs to take on a communications role with employees and customers," he says, which are just as critical to controlling perceptions that a business will be rapidly resuming normalcy."

For many banks, exposure to pandemic-related desktop exercises may have been the illumination of a professional lifetime. It focused them on the scenario building that's necessary to both formulate and run through a decision tree that could take a bank under siege through emergency response, and several phases of recovery.

Business Continuity/DR RESOURCES

* Business Continuity Institute/DR (www.thebci.org) comprehensive educational resource

* Continuity Central (www.continuitycentral.com www.continuitycentral.com/feature0298.htm) includes specifics on testing

* Department of Homeland Security FEMA (www.fema.gov) includes information on BC/DR programs

* Disaster Recovery Institute International (www.drii.org/DRII/) comprehensive educational resource

* Pandemic Resources (www.pandemicflu.gov)

* Vital Records Protection (www.vitalrecordsprotection.org) comprehensive resource for records protection issues, guidelines

* U.S. Dept. of Commerce, Office of Security/Seattle (www.wasc.noaa.gov/wrso/oep-coop.htm) includes samples of occupant emergency plans

WMD Consulting (www. wmdconsulting.us) terrorism vulnerability assessment

Source: Rothstein Associates, Inc.

Business Continuity/DR TRAINING/CERTIFICATION

* American Bankers Association/ICB (www.aba.com) Institute of Certified Bankers certification, Certified Financial Service Security Professional; includes course work in BC, DR, and emergency response.

* Association of Contingency Planners (www.acp-international.coml) offers various educational programs.

* Business Continuity Institute (www.thebci.org) BCI Basic Certification based on demonstrating "Common Body of Knowledge" and having appropriate professional experience. BCI also offers Advanced Fellow grade certification.

* Disaster Recovery Institute International (www.drii.org/DRII/) Basic certification based on demonstrating "Common Body of Knowledge" and having appropriate professional experience. BCI also offers advanced Master Grade certification.

* International Association of Emergency Managers (www.iaem.com/) Offers Certified Emergency Manager (CEM) program.

* Various universities offer Associate, Bachelor, and Post Grad programs. (www.disaster-resource.com/cgi-bin/ article_search.cgi?id='62') Article lists various programs.

Source: Rothstein Associates, Inc.

By Lauren Bielski, senior editor