Missing and Elusive Metrics

By Rees Morrison

GC New York

June 19, 2008

Those who conduct surveys and publish law department benchmarks have for years pounded and pounded on the same metrics. The same handful of key metrics has become enshrined, but it is time to think more deeply and advance to more sophisticated data collection and analysis. Not that the current crop has not pushed general counsel to think about measures of productivity and cost-effectiveness. Indeed, many general counsel do so, and to compare a department's spending and staffing figures with the comparable figures of other law departments is a long step toward realizing where to improve.

Still, progress urges the law department field to tackle metrics of the kind described in the first part of this article. The top executives of companies constantly push for efficiency improvements, so law department managers need to devise better analytical tools. To put my main point generally, we need more benchmark findings on correlations between management practices and financial results. To facilitate getting to that point, general counsel need to speak out on the value of better metrics and, as importantly, fund or participate in forward-thinking benchmark research.

Awash in the customary benchmarks, general counsel feel parched about the lack of other important benchmarks. Many general counsel care about the commonly produced benchmarks, but also wish there were meaningful quantitative guidelines for many other areas of law department management. This article looks at a selection of law department metrics that should be developed and then turns to a group of metrics that will probably never be available. The article's goal is to stimulate law departments and service providers to fill the gaps by collecting somewhat different metrics and searching for correlations between management actions and effectiveness.

Productivity boost of secretaries and paralegals. With enough data, it would be possible to show whether and how much secretarial support enables law departments to be more productive. By productive, I mean providing industry-standard legal services at least and at a total legal cost that is better than the median figure for representative companies. We know typical ratios of so-called admins to lawyers, but we do not know the figures to attach to their contribution. Likewise, no one has shown numerically the effect of different ratios of paralegals.

The industry needs a large enough data set - probably 400-plus law departments - so that the analysis can show statistically whether changes in non-lawyer support pay off. Ideally, the results would show that more support enables the lawyers to serve their companies more effectively even though the inside budget increases.

Substitution of paralegals for attorneys. Much is written about the importance of in-house counsel delegating work to paralegals. Even so, it is my belief that most law departments could use more paralegals. We lack quantitative evidence to support conclusions about the right number of paralegals in the various practice areas and how lower-cost but experienced paralegals might take over portions of work now done by lawyers.

To test this hypothesis, we would need to know years of experience of paralegals and their areas of practice focus. Then, with sufficient data, we could draw conclusions about something akin to a substitution rate of paralegals for lawyers by practice area.

Practice area benchmarks. Nearly all of the available metrics treat law departments as a homogeneous whole. Total legal spending as a percentage of revenue, for example, is rarely broken down into a comparable figure for litigation lawyers, commercial lawyers, environmental lawyers and other specialties. Nor do we know such things as labor lawyers per 1,000 employees under collective bargaining agreements; Superfund sites per environmental lawyer; or corporate secretary staff per corporate entities maintained.

Benchmarks for practice groups can be developed and would be of much benefit to general counsel. The path forward here is to combine measurable output or task data with numbers of lawyers and thereby generate comparable performance metrics. For example, securities lawyers might be matched against Securities and Exchange Commission filings per year together with other measures of productivity. To do this we will need standardized understandings of work and output.

Baseline spending. Spending figures for law departments tend to be spiky. One or two large cases or transactions drive up the external spending numbers for a year and conceal the underlying, business-as-usual spend numbers. Managers of law departments do not have a standard way of defining one-off expenditures, and therefore we lack industry-wide benchmarks on what remains: core spending. Just as the Consumer Price Index looks at the less-volatile core spending, so should law department managers and the benchmarks they rely on.

We need definitions of highly unusual matters, or a way to think about consistent treatment of unusual spending. If law departments broke their data on spending into those one-off oddities and also into ongoing, core spending, our understanding of spending patterns and correlations would deepen significantly.

Caseloads effectively managed by litigators. Lawsuits vary enormously in their complexity and time demands. For that reason, the raw number of cases any individual law department oversees offers hardly any insights. If, however, enough law departments contributed data about caseloads by broad categories of cases, such as all HR-related cases, we might be able to come closer to understanding typical workloads. When we couple data on caseloads by category with spending on litigation as a percentage of revenue, we will draw closer to a useful benchmark metric.

The five missing metrics described above are an illustrative subset of many more statistics that would be informative for those who manage in-house lawyers. I chose them because it is realistically possible to collect the data; the findings from them will be instructive; and thoughtful general counsel could use them to make decisions more wisely. As important, I believe these metrics could be shared by companies without undue risk, gathered in sufficient numbers to be statistically reliable, and correlated with familiar metrics. If much of this were to happen, the legal industry would become more efficient.

Metrics Not Possible

As important as it may be to push the envelope on metrics, it is important to know what cannot be expressed in probative numbers. The four metrics I am about to describe, by contrast to the earlier set, will never be available. True, we who are involved in the management of in-house legal teams sorely miss these metrics, but this thirst will never be sated.

Legal risks avoided in terms of dollars. Sadly, one major contribution of a well-run law department is to reduce or avoid legal risks. Ironically, there is no way to measure risks avoided. We can't quantify this known unknown. Our only partial hope is to compare such metrics as lawsuits pending, external spending, and settlement figures of companies that chose one risk avoidance approach compared with comparable companies that chose a different risk-avoidance approach. The logistics of such a benchmark survey are daunting, and the conclusions would never be free from disagreement and attack. Thus, this all-important metric - the value of keeping a company out of legal trouble - will remain elusively beyond our reach.

Return on investment for management initiatives. Every day, general counsel start out confidently on management initiatives. They decide to install a matter management system or start a knowledge management initiative. Perhaps they decide to push cost control measures or increase diversity. Whatever the initiative, the return on amounts spent in the undertaking can never be provably amortized by savings that result. Vendors and consultants can conjure up imagined cost reductions or productivity gains, but the reliability of those findings are highly suspect. The reason is that the law department cannot run along a parallel track and compare the before-initiative outcome to the post-initiative outcome. It is sufficient that good managers rely on their belief that doing something new is better than the status quo.

Index of complexity of legal work. We hear unceasingly about commodity legal work. We believe we can spot run-of-the-mill legal services, but no one has devised how to quantify to what degree a given legal situation is more or less complex than another. The complexity of matters worked on by in-house counsel is, to put it simply, too complex to quantify. Yes, we all think we know it when we see it, but to be able to translate attributes of a matter into a standardized assessment of its complexity has so far eluded us, and will continue to evade us.

Measure for law department productivity. For several years I have written on my blog about many aspects of law department productivity. Indeed, as a consultant, I advise general counsel how to produce more of equivalent quality with the same or fewer resources. That is productivity, but what we cannot pin down is a way to measure and benchmark comparative law-department productivity. To the same point, we have no way to express in figures how productively groups of lawyers work. In part this is because counsel given in meetings or over the phone is ephemeral. Legal output remains a chimera. It would take survey participation numbers far greater than any that have so far been obtained for us to even take reasonable steps to understand productivity drivers. For the picture to be made complete in numbers, we have eternity to wait.

Conclusion

Perhaps, with the accumulation of more advanced metrics like those described at the start of this article, especially if we can show correlations, we will come closer to understanding some of the unobtainable, metrics described in the second part of this article. We might even bravely try to measure teamwork within law departments and its effect, if any, on productivity and quality. Oh, and quality can't be measured either. But we can and should push further into the available and useful numbers that relate to law department management.


Rees Morrison, president of Rees Morrison Associates, helps general counsel manage legal departments. He hosts the blog www.LawDepartmentManagementblog.com and can be reached at Rees@ReesMorrison.com.