Compliance &Ethics

Compliance & Ethics January

2015

Professional

a publication of the society of corporate compliance and ethics

www.corporatecompliance.org

Meet Barbara Harmon Compliance and Ethics Program Lead Alyeska Pipeline Service Company See page 16

23

Some realism about risk assessments Scott Killingsworth

35

The evolving role of the chief compliance officer Patrick Quinlan

39

Measuring Jim McGrath Thomas R. Fox

41

The ends do not justify the means: Performance management and ethics Ruth Steinholtz

This article, published in Compliance & Ethics Professional, appears here with permission from the Society of Corporate Compliance & Ethics. Call SCCE at +1 952 933 4977 or 888 277 4977 with reprint requests.

FEATURE

by Scott Killingsworth

Some realism about risk assessments »» A one-size-fits-all risk assessment fits no one well; tailor your assessment to fit your own company and your top objectives. »» The one imperative of a risk assessment is to drive improvements in your compliance program. »» Resource trade-offs are inevitable; size the assessment to your resources and when in doubt, favor mitigation over rumination. »» Technology tools can generate vast amounts of data easily, but beware of data overload, orphan findings, and false precision. »» Big data isn’t always better data; quality counts, expertise matters, and insight is the goal.

But the organizational contexts in which these are applied are as individual and varied as the mutts in a pound. What works for a purebred Great Dane is not going to be right for a Labrador puppy, much less (these days especially) for a Pekinese or Russian Wolfhound.

What works for a purebred Great Dane is not going to be right for a Labrador puppy, much less (these days especially) for a Pekinese or Russian Wolfhound. In “foundational” or first-time risk assessments, it’s not rare to see the assessment team set out confidently, armed with another company’s templates, tools, and process plans, only to stall, or veer off with mid-course corrections, when it becomes evident that +1 952 933 4977 or 888 277 4977 www.corporatecompliance.org

Compliance & Ethics Professional January 2015

S

tephen Covey was onto something when he came up with his second habit for effectiveness: “Begin with the end in mind.”1 Nowhere is this principle more important than in a compliance risk assessment, where effectiveness depends on correctly balancing methods, resources, and results. Focusing first on defining clear objectives helps us make sound judgments – both in advance and later, on-thefly – about how efficiently a given activity, expenditure, or method Killingsworth will contribute to achieving our goals. Equally important, thinking carefully about the desired end-point forces us to consider what outcomes are realistic and achievable, given the available resources and time. The basic elements of risk assessments are well-known: information gathering on key risks and the associated likelihood, impact, and controls, gap analysis, and design of remediation or mitigation steps.

23

FEATURE

differences in company attributes demand a different approach. Anyone planning a compliance risk assessment certainly must take into account gross variations in company attributes that affect both the risks to which the company is vulnerable and the assessment processes that will be practicable—characteristics such as size, geographical presence, internal authority structures, number of employees and revenue per employee, number of business units and product or service lines, etc. Every industry is different; the compliance risk profiles of General Electric, General Mills, General Dynamics, and General Motors defy generalization. A company’s growth history is equally important; for example, roll-ups and conglomerates have quite different structures and issues than companies that have grown organically, and mature companies have different risks than those still in an adolescent growth spurt. These factors will dictate some contours of a risk assessment and rule out others. The permutations are endless, and it’s impossible to write a prescription that would cover them all. My modest aim is to identify just a few considerations about purpose, process, and practicality where experience has shown that a little reflection on the front end can substantially increase the assessment project’s efficiency, that is, the ratio of effort to useful output. I’m interested in what is knowable about risk, how much we need to know in order to map out effective action, what effort is required to know it, and what processes will minimize that effort. I’m skeptical about the

quest for perfection and vitally interested in the point of diminishing returns.

The objective The fundamental objective of a compliance risk assessment is to drive improvements in the compliance program. It should produce information to guide the allocation of resources and activities within the compliance program so as to optimize the match between the company’s greatest compliance threats and the corresponding mitigation efforts. Specifically, we want to: ·· First, identify and prioritize the material, inherent risks of our business; ·· Second, assess existing controls for those risks and prioritize our residual vulnerability to them; and ·· Third, apply this knowledge and these priorities to reduce the targeted risks, by changing business processes; providing improved training, policies, or access to advice; adjusting incentives; assigning accountability; or establishing or enhancing monitoring and controls.


Every industry is different; the compliance risk profiles of General Electric, General Mills, General Dynamics, and General Motors defy generalization.

24 www.corporatecompliance.org +1 952 933 4977 or 888 277 4977

Anything more is either a luxury or a misallocation of resources, depending on your viewpoint.

Tradeoffs are a reality After all, compliance resources are finite, and devoting resources to one project ordinarily means withholding them from another. A risk assessment that is too elaborate or too detailed (i.e., one that produces more information than can be profitably analyzed, or more analysis

FEATURE

than can be acted on while still relevant) steals resources from mitigation efforts as well as from the capacity to perform day-to-day compliance tasks. It’s not just a resource issue. Producing much more risk-vulnerability information than can be put to timely use leaves a paper trail of futility and may set up the risk owners for a record of perennial failure, or even for accusations of neglect. One justification for elaborate risk assessments is to impress regulators if leniency is ever needed. In a world without resource tradeoffs, the point may be welltaken. But consider an enforcer’s perspective in the wake of misconduct that your compliance program failed to prevent. Would they rather see that you carefully identified and analyzed your top ten risks and rolled out robust corrective action plans for five of them, or that you obsessively analyzed your top 50 risks, but only had time to establish action plans for three? I believe the compliance program with greater mitigation activity will evoke more sympathy than the one that knows more, but does less.

that cannot be used before it goes stale. The following are some things to think about in sizing the assessment. The risk universe How many different risk areas should be assessed? In most cases, a small group of legal and compliance experts triages a comprehensive, “kitchen sink” risk list down to a manageable number to examine more systematically. In addition to their basic knowledge of the legal issues applicable to the company’s activities, the triage uses other aspects of this group’s “tribal knowledge,” such as past compliance problems in the company or the industry, recent regulatory activity, the company’s authority structure and culture, and a general sense of the existence or maturity of controls in different areas. How many risks are “manageable” to assess fully in a given company is open to wide variation, particularly based on the rate of change in the company’s business environment. For example, a mature company in a stable industry with little regulatory flux might justifiably conduct a very broad foundational risk assessment, in order to generate a wide library of baselines to be addressed over the next several years. But imagine a dynamic enterprise with a fledgling compliance group that believes 80% of the company’s compliance risk probably resides in five areas. This company might want to look pretty carefully at ten risk areas, with the goal of validating or correcting the initial risk assumptions, remediating the eventual top five in the first year, and addressing the next tier later.

Adjust the size with your eyes on the prize If the end we have in mind is improving the compliance program, then we should begin by sizing the assessment with an eye towards the resources that will be available over the near term for analyzing controls and designing and implementing corrective plans. Businesses and their regulatory risks change over time, and the point is to avoid investing heavily in generating large amounts of information

+1 952 933 4977 or 888 277 4977 www.corporatecompliance.org


How many risks are “manageable” to assess fully in a given company is open to wide variation, particularly based on the rate of change in the company’s business environment.

25

FEATURE

Aside from the concern about multiple “orphan” findings that cannot be acted upon while still fresh enough to trust, we should consider that there is some incremental cost associated with every additional risk area examined. Much of the cost of a risk assessment is the time participants outside the compliance staff must take away from their “day jobs” in order to provide input. This has obvious costs to productivity, but also a hidden cost in using up some of the Compliance department’s limited claim on employees’ time— time that might be put to better use in compliance training. It’s also worth emphasizing that risk assessments seldom result in a substantial re-ordering of the risks initially assumed to be in the top tiers of materiality— big surprises are rare. The greatest actionable value in an assessment usually emerges from the review of controls in the context of business processes associated with these major risks. A slightly different risk-universe issue is whether there are categories of compliance risks that can appropriately be carved out of the scope of an assessment, typically because they are being carefully examined in another context, are considered beyond the scope of the Compliance function, or both. Some companies carve out Environmental or Safety, for example, if those functions already house robust risk assessment and remediation efforts of their own. Similarly, Sarbanes-Oxley

internal controls risks, tax risks, and the technological side of data security risk are often scoped out of the compliance risk assessment process. Many companies exclude contractual breach risk from their assessments (and from Compliance’s purview generally). Another approach for distilling down the risks to be examined is simply to plan separate future assessments of specific areas, especially risks that might benefit from a different methodological approach. For example, a foundational risk assessment might focus on “hard,” substantive regulatory risks, in anticipation of a later assessment focusing on more generalized bellwethers of misconduct such as organizational culture (which requires data about employee perceptions), or incentives and compensation structures. Similarly, assessments of thirdparty relationship risks; risks associated with acquisitions, divestitures, and reorganizations; and “deep dives” into a particular country’s risks would likely benefit from a distinct methodology, different participants, and a separate process.


Aside from the concern about multiple “orphan” findings that cannot be acted upon while still fresh enough to trust, we should consider that there is some incremental cost associated with every additional risk area examined.


Technology tools and human bottlenecks The availability of sophisticated technology tools or input forms can make it cheap and easy to gather voluminous data from a large population, and to conduct statistical analysis of that data. Potentially, this means we can conduct a much larger assessment with the same resources. There is definitely a place for crowdsourcing compliance anxieties as

FEATURE

well as casting a wide net for particular fact patterns of concern, such as use of thirdparty sales intermediaries or maintenance of consumer personal data. But “big data” is not always better data. The essence of a good risk assessment is not popular opinion, mechanically sliced and diced; it is informed opinion and expert judgment applied to the facts. Not only should we beware of gathering far more data than we can follow up on; we should beware of gathering more than we will be able to analyze meaningfully at the human-judgment bottlenecks in the process. We shouldn’t ask everyone about everything.

of large numbers and yield information of questionable quality. It is true that as sample size grows, the survey results will get ever closer to the true average opinions of the entire population from which the sample is drawn. But the important question for our purpose is whether the average opinion of a broad employee population about compliance risk actually gives us better insights than we can get from a smaller but more carefully selected sample.

Who will we ask? Ordinarily, risk assessments gather information from senior executives, senior members of staff departments (e.g., Legal, Compliance, Human Resources, Information Technology, Internal Audit, Controller, Environmental Health & Safety, Finance, etc.), as well as a sample of senior operational personnel in the business units. To the extent that “risk owners” are not in these groups, they are usually sought out, and sometimes manager-level input is requested as well. How many people, and who, should participate depends on the risks being assessed, the required timeline (scheduling difficulties increase with the square of the number of people required in a meeting), and the methods of inquiry. One-on-one interviews and focus groups can uncover fantastic information, but may seem costly compared to surveys, particularly in a very large organization. Surveys can reach more people faster, but may defy the law

How will we ask them? Especially with surveys, experience suggests that the more questions you require people to answer—particularly questions not intimately related to their jobs—the less effort and the more guesswork will go into the answers. Lower-quality data with a high “noise” content is a predictable result of plumbing the depths of employees’ blind spots and attention spans. Unfortunately, customizing surveys by department, job function, or risk area can be costly. To deal with this dilemma, one useful precaution is to ensure that the survey is done in a way that allows the data to be sorted and analyzed according to business unit, department, location, etc. This way, a little filtering of the results on the back end might produce much clearer and more consistent information. Another way to reduce the static of risk illiteracy in the data is simply to include a question about the employee’s familiarity with each risk area, or to offer some form of “don’t know” option, rather than forcing everyone to assign likelihood and impact values to every risk. +1 952 933 4977 or 888 277 4977 www.corporatecompliance.org


Surveys can reach more people faster, but may defy the law of large numbers and yield information of questionable quality.

27

FEATURE

Another tack is to survey on key subjective questions that each employee, by definition, knows the answer to: Do you feel you adequately understand the legal requirements applicable to your job? Are you able to get the information and advice you need in order to follow the law? Do you feel that integrity is an absolute at our company? And participants should always be asked open-ended questions such as “What else should we be asking about?” Or “What keeps you awake at night that has not been discussed?” Or simply, “What would Satan do?”

Next, a numerical value is assigned to the totality of controls for that risk (including policies, procedures, training and communications, supervision and monitoring, approval structures, “hard” detective and preventive controls, etc.), and this value is used to derive the residual risk. Sometimes the susceptibility of the risk to additional controls is then used to prioritize remediation. All of this looks great on a bubble chart or spreadsheet, and it is a helpful way of visualizing and prioritizing risks, but the apparent precision of numbers and graphs can be deceptive. There is a substantial margin of error in each number, one that can be exacerbated by many different factors, including: ·· an unwillingness to acknowledge (or “motivated blindness” about) the risks of one’s own activities, ·· an inability to recognize or evaluate the risks of operations we are less familiar with, ·· a flawed understanding of the relevant law (which is of course a risk factor in itself), ·· incomplete awareness of all of the relevant impact factors for a given compliance violation, and ·· the imponderability of reputational damage and its effect on customer relationships.

We all love certainty, precision implies certainty, and numbers imply precision. But to paraphrase the inscription on your passengerside mirror, “Objects in spreadsheets are fuzzier than they appear.”


The lure of precision We all love certainty, precision implies certainty, and numbers imply precision. But to paraphrase the inscription on your passengerside mirror, “Objects in spreadsheets are fuzzier than they appear.” Risk, after all, is a function of uncertainty and any attempt to precisely quantify the unknown involves a hint of paradox. In considering the methodology for your risk assessment, a little realism about the precision of numerical inputs and outputs can go a long way towards getting useful results at a reasonable cost. Risk assessment questionnaires and focus groups commonly require the respondents to categorize each risk into one of several low-tohigh likelihood categories. Similar categories are assigned for impact. Each category is assigned a numerical value (e.g., between 1 and 5), and the likelihood and impact numbers are multiplied against one another to yield a composite index of inherent risk.


Consider judgments of likelihood: How many people can really wall off their awareness of (and opinion about) existing controls when rating the “inherent likelihood”

FEATURE

of a breakdown? Might this artificially depress the likelihood number for some risks? Harvard’s Thomas R. Powell famously said that “If you can think about something that is related to something else without thinking about what it is attached to, then you have what is called a ‘legal mind’.” Not everyone can do that, including lawyers.2 Consider judgments of impact: How many people in your organization understand the risks of False Claims Act litigation, debarment proceedings that may be contagious across governments, breach-of-contract damages or contract terminations based on compliance violations, treble damages for bidder collusion, and loss of licensure? How many would consider all of these when assigning an impact value to the government contracting risk area? In a recent compliance risk assessment seminar, the audience was polled about the biggest challenges they faced in performing risk assessments. Number one was developing a consistent means of measuring compliance risk across the organization—the apples-andoranges problem. Number two was getting sufficient time, engagement, and input from functional business leaders. Think about that. You can’t get key people to pay enough attention to give you their best input, and what input you do get, you don’t know how to compare in a way that lets you rank the different risks with confidence. These two problems are not going away. Some try to address the apples-and-oranges issue with fine-grained definitions of each

level of likelihood (X will happen once in every how many months or years?) and impact (dollar amounts for each level), and maybe that helps. But are answers to these questions really likely to be any more accurate than “low,” “medium,” and “high”? And attempts to deal with the comparability issue may actually exacerbate the attention issue. In any but the most motivated respondent pool, multiplying the number of questions and sub-questions, and providing detailed instructions about what factors to consider, may simply lead to questionnaire fatigue, glazed eyes, and disengagement. As must be apparent, I don’t believe that asking a lot more people significantly improves the accuracy of the answers, unless those people actually have significant incremental knowledge. You can’t reinforce a house of cards by adding more cards. Those of us who predate the handheld calculator were taught the principle of ”significant digits” as part of learning how to use a slide rule. It says that the outcome of a calculation can’t be any more precise than the least precise number used in the calculation. If you multiply 4 times 1.3572, and your 4 is only accurate to the nearest whole number, the answer you’re allowed to use is not 5.4288, it’s 5. The .4288 is “spurious precision” and means exactly nothing. Let’s apply this principle to those likelihood and impact numbers, and all of the guesswork that goes into producing them. Suppose your survey shows a risk likelihood of 2 on a 10-point scale, but because of the +1 952 933 4977 or 888 277 4977 www.corporatecompliance.org


Harvard’s Thomas R. Powell famously said that “If you can think about something that is related to something else without thinking about what it is attached to, then you have what is called a ‘legal mind’.”

29

FEATURE

margin of error, the underlying reality might actually be a 1 or a 3. Similarly, you have an impact rating of 5 that perhaps should be a 4 or a 6. When you multiply them, your spreadsheet shows the risk as a 10, but your range of potentially correct risk indexes is anywhere from 4 to 18. That’s quite a swing.

Putting it all together: The indispensability of judgment So, what can we conclude about the issues of who and how many people you should poll in the first place, how much effort you should expend (or require your respondents to expend) to generate numerical data to crunch, how much confidence you should have in the crunched data, and who should make the final assessment and prioritization of risk? I believe that interviews and focus groups produce better information than written surveys, and that surveys with a few openended questions, or that invite the respondents to explain their conclusions, are usually better than purely numerical surveys. I do think that surveys are valuable, but in most cases, they are just a starting point for analysis. They tell you where to look closer. So think twice before committing a lot of resources to expanding and fine-tuning surveys and, especially, requiring larger and larger groups to answer them. I believe that for most organizations, an expert-driven process will yield the best results at the lowest cost. I believe that subject-matter experts should triage the original list of risks to be assessed, should vet the questions to be asked, should independently review the existing control environment, and should

make the final assessments—after considering everyone’s input, but without according survey numbers any talismanic power. Of course not all surveys are alike. There are some incredible instruments out there. I have seen detailed worksheets for each risk area that list multiple risk-specific, fact-based indicators for both likelihood and impact, with numerical weightings for each. The key is that these factors and their weightings were thought out by experts who understand the business, the relevant law, and the potential impacts of violation. But few companies have the scale to afford a custom instrument that is defensible to this level of detail, or the will to ensure that employees invest the necessary time and effort to answer all of these questions carefully. And it is still fair to ask whether such a granular assessment leads to a materially different risk ranking, or to more effective program improvements. The rest of us can take comfort in the fact that even though our inputs are inexact and our conclusions debatable at the margins, any thoughtful risk ranking, controls evaluation, and gap analysis by knowledgeable people will be a big step forward, as long as it leads to action. We need to exercise judgment and accept accountability for it. We are professionals, and that’s what we’re paid to do. ✵


I do think that surveys are valuable, but in most cases, they are just a starting point for analysis. They tell you where to look closer.


1. Stephen R. Covey: The 7 Habits of Highly Effective People. Free Press, 1989. 2. Quoted in Thurman Arnold: “Criminal Attempts: The Rise and Fall of an Abstraction.” Yale Law Journal, 1930, vol 40, pp 53-80.

Scott Killingsworth ([email protected]) is an Atlanta-based attorney and Partner at Bryan Cave LLP. He assists clients with development, implementation, and evaluation of ethics and compliance programs.