Rees W. Morrison is Guru for Online, Law-Related Surveys and was  a principal of Altman Weil, Inc. He has more than 25 years of experience advising law departments on cost control, department structure, process improvement, outside counsel management, performance benchmarking, and other key issues. He also specializes in data analytics for legal organizations.

   Before joining Altman Weil, Mr. Morrison consulted independently for five years and held partnerships at several legal consulting firms, including an earlier tenure at Altman Weil from 1998 to 2002.  He has had several in-house positions including Business Manager for Google’s law department and Consulting Assistant to the General Counsel of Merck. Earlier in his career he was vice president of two software firms, and an associate at Weil Gotshal & Manges and two other New York law firms.

   He has written extensively on law department management, including nearly two-hundred articles, six books, and a well-known blog on law department metrics. For two years he wrote a bi-weekly column, Morrison on Metrics, for InsideCounsel. Among his books are “Law Department Benchmarks: Myths Metrics and Management”; “Client Satisfaction for Law Departments”; and “Law Department Administrators: Lessons from Leaders.”
   He is a Certified Management Consultant (CMC), a member of Scribes (The American Society of Legal Writers), a fellow of the College of Law Practice Management, and has been on the Board of Advisors of legal publications Corporate Counselor, Law Department Management, and Metropolitan Corporate Counsel. A Life Fellow of the American Bar Foundation, he has participated in the ABA's Law Practice Management Section and ACC’s Law Department Management Committee. 

   Mr. Morrison graduated from Harvard College in 1974, earned his law degree from Columbia Law School (1978) where he was a law review editor, and received an LLM from New York University Law School (1984)

_________________________________________________________________________________

Day after day, law firm partners and managers confront operational problems, think about them, and make choices about what to do or not to do. In other words, they decide something. They would make better decisions if they took into account the data available to them. If they collect metrics and weave them into their deliberations, the outcomes will be both sounder and easier to explain.

Two subtler and broader advantages from decision-making that incorporates numbers should be emphasized. First, it encourages a different way of thinking about decision-making than traditional approaches. Make it a practice throughout the firm to ground arguments in data and to present arguments buttressed with numbers, or else accept that a resolution rests on power, values, or ideology more than quantifiable evidence.  

Second, being mindful of data is being mindful of what you do. This is a deeper benefit arising from a law firm’s receptivity to data. A general awareness of metrics helps lawyers and others in law firms take stock of their processes, describe them and their output in more tangible, numerate terms (“15 10Ks reviewed this month” rather than “Lots of 10Ks”). They become more aware and reflective about what they are doing and how they might do better.

But let’s consider our claim — “data helps decision-making” — and ground it by walking through a scenario for a management decision. Assume that three partners want to figure out whether to hire another paralegal. Further, assume that the partners disagree. Their decision will improve if they marshal relevant data, analyze it effectively, and apply it to their discussion. How can numbers help them objectively think through the problem and potential solutions better than they would have without? Here are five ways.

1.   Sidestep Cognitive Fallacies. 

Data can counteract many of the cognitive biases that afflict decision-makers. Often we are unaware of the gremlins in our minds that attack what we believe to be our clear-headed, balanced evaluations. Consider four well-known cognitive fallacies and how data might correct for them:

Framing: An antidote to framing could be benchmark data on paralegals per lawyer in firms.

Salience: To blunt its potential impact, someone could gather articles that report average lawyer/paralegal ratios based on surveys of many companies.

Confirmation bias: Perhaps the partners’ practice group submitted the mixed evaluations on a survey of paralegals, which would be data that challenges a one-sided view.

Risk aversion: A risk-averse partner may argue for more paralegals because the group never wants to be over-extended; past data on large bumps in hours might dispel the concern.

2.      Uncover and Query Empirical Assumptions.

When people make decisions, they often neglect to articulate the factual assumptions on which they base them. Worse, they may not even realize that they have been motivated by unstated (and usually untested) beliefs about how common something is or how much there is of something measurable. For example, one partner might accept on faith that lawyers will delegate work to paralegals, while another could trust without verifying that it will be easy to find, hire, and retain capable paralegals. If underlying assumptions such as these are not identified and if there is no data either way, decisions will likely be weaker (and take longer).

3.     Disrupt Entrenched Convictions.

As data becomes available for decision-makers, they should incorporate it and change how they view the probability of being correct. In a Bayesian view (a fundamental technique of statistics), new data changes what are called priors and helps make predicted outcomes more accurate. New findings and facts should cause thoughtful people to recognize and reconsider their animating beliefs.

4.     Delay Premature Conclusions.

When making a decision, it is crucial not to seize upon the first plausible solution to the problem. Much better, instead, is to keep exploring alternative possibilities. Data helps stimulate new possibilities to address a problem or to encourage managers to think about the problem longer.

5.     Counter Peer Pressure.

In groups, solid data can serve as a talisman against the fancy (or loutish) talker, the zealot, or the high-ranking executive. A partner who disagrees with her colleague might be more inclined to back them if some metrics bolster the point.

This is not to claim that good data automatically means good decisions. It is to claim that operational data can frequently help steer managers to reach a sounder decision. To be sure, the toughest decisions tend not to have decisive metrics. But even the gnarliest decisions — those that entangle personalities, tradition, long-term visions, or fights over fundamental values — can benefit from whatever dollops of data are available.  

1.       What Data Do Law Firms Have That Can Contribute to Better Decisions?

Analytic tools require data, and law firms have data sets aplenty. From this author’s recent book, “Data Graphs for Legal Managers,”[1] the table below shows a wide array of firm data.

A.     Types of data law firms have.

Starting with such internal data, firms can mix in information from other sources, possibly within the firm or from data outside the firm. As for client information, supplemental information might include whether it is publicly traded, whether it has an in-house law department, its revenue, the number of employees, SIC codes, and more. For example, you might want to combine time and billing data or input information from your HR system. Years of experience or academic degrees of timekeepers could be mixed in from the firm’s HR database. 

B.     Repositories of data.

All this potentially useful data lives in paper files, people’s memory, time and billing software, customer relationships software (CRM), marketing records, personnel files, exhaust (data that is created by someone doing something else, like making phone calls or scheduling conference rooms), general ledgers, status reports, surveys, and counts of all kinds of events. A law firm that wants to capitalize on data needs to extract and store it in databases, spreadsheets, in the cloud, or wherever it can best make use of it.

C.      Data cleaning.

            A key part of data analytics is the unglamorous slog of grooming it for analysis.  Software can help users correct the data, such as when there are missing or highly unusual values, the latter being what data analysts call outliers.

Clean data also cannot have too many missing values or different styles in cells of the same column. If the fees column has cells with a dollar sign and other cells without the sign, for example, the software might stumble. If some cells in the spreadsheet show “NA” when data is not available, but others show “—“, you need to clean that. Clean data is also reasonably accurate data (it was not a data entry error that some associate billed 4,299 hours last year!) and not pockmarked with bizarre values.

Still, you can actually do useful analyses and, more fruitfully, make predictions with modest amounts of data. For example, with a spreadsheet having details on 50 or more closed cases or matters of a similar type, you can infer a great deal.

2.     What Are Potential Uses of Data Analytics by Law Firms?

Metrics and their exploration can benefit law firms everywhere that partners contemplate management decisions. The next section sketches several decisions that could benefit from data analysis and metrics.

A.     Increase Revenue.

The recent surge in law firms collecting, analyzing, and visualizing information aims —quite understandably — to increase firm revenue. Why, managing partners ask, should we invest the time and money to do predictive analytics (AKA machine learning) if we don’t expect to hear the cash register ring? That goal of increased fees (or improved profitability) makes sense. It also orients firms to focus analytic tools on substantive legal analyses. Much can be done to transform the straw of data into the gold of profitable clients, practice groups, or billing arrangements. Analyzing cost drivers of lawsuits to make more money on fixed-fee arrangements would be an example.

B.     Retain and Wisely Promote Associates. 

One benefit of data is when the firm is hiring lawyers. When firm ambassadors make their pitch to hire associates or lure lateral partners, they deserve to be able to describe the firm glowingly and convincingly. Solid, impressive numbers on growth, revenue, quality, and associates, not to mention clients, persuade recruits, especially when made clear with effective graphs. Or they will let machine-learning software loose to study who makes partner and why, or to tackle attrition in terms of which desirable associates are at risk of leaving the firm.

C.     Improve Firm Operations. 

A number of benefits of predictive data analytics should be recognized in the domain of law firm operational management. As much as managing partners want to grow or increase profitability and bring in more fees and add more lawyers, they may overlook or discount secondary uses of law firm data for running the firm as leaders focus almost exclusively on the short-term return on investment in business development.

Facilities. Another use of data arises frequently in infrastructure planning. Should we sublet additional space? Should we move to another location or open a branch office? Sometimes there are questions about installing a larger server or rewiring the existing offices. Answers to questions like these, and decisions made thereafter, are wiser when there is data available to support them. 

Proposals. Almost every Request for Proposal that a firm receives asks for data. The law department that issued the RFP wants to know about diversity, or about practice groups and their numbers of lawyers, or about the size of transactions handled recently. It is efficient to have the raw data already compiled and curated in a spreadsheet or database. 

Press Relations. When reporters call, the partner who responds will make points more tellingly if they can rapidly cite reliable facts about the firm or topic. “Almost 40 percent of our clients do business in more than 10 countries” impresses reporters far more than getting back two days later with “Lots of our clients are multinationals.” The first statement, with its impressive precision and prompt delivery, can only be made if the appropriate numbers have been tracked, analyzed, and made available. 

Vendors: Any time a law firm considers buying something, it will make sounder decisions if it precedes the decision with tallies and tracking. Do we need to buy more user seats under a software license? Have people made sufficient use of the expensive subscription? Research into these kinds of questions pays off; research should be captured as data for decisions.

Law firms focus on data associated either with client matters, or with the effective deployment of their own lawyers and staff. They won’t regard their spending on vendors as nearly as vital as matter productivity, investment, and outcomes.

3. What kinds of analytical tools are available?

Some partners in law firms may not be aware of the full panoply of data analytics that their firm might employ. Let’s briefly review eight different analyses that software can produce.

Descriptions and summaries of data.

At the most basic level, software can take data, such as the billable hours of lawyers, and describe with varying degrees of summarization the key numeric features of the data. Software can calculate the average billable hour, the median of the billable hours, or how dispersed it is (usually expressed as standard deviations). Software can pick out the lowest value and the highest, break them into groups (called quantiles), and tell us ranges. Contingency tables can also illuminate the data. Furthermore, software can depict those features of the data in graphics, such as histograms, density plots, scatterplots, and bar charts.

Correlations between variables in the data.

It may be useful for legal managers to see how one variable (an element of data tracked for every associate, client, lawyer, or whatever) moves up or down in relation to the average when another variable changes. Thus, for instance, the software can show the correlation between the number of matters worked on by a lawyer and billable hours reported. A correlation tells you whether there is an association between two variables, how strong it is, and in what direction the variables move. It is a positive correlation if both numbers move in the same direction (such as higher billable hours and higher bonuses); it is a negative correlation if the numbers move in the opposite direction (such as higher billable hours and lower psychological well-being).

Comparisons of averages and differences.

Several statistical tools can detect whether the difference between two or more numbers has significance mathematically. So, for example, there are many techniques to tell whether the average billable hours in a year between two offices of a law firm vary enough for managers to consider intervening and taking some action. These tools, such as ANOVA and the Student’s T-Test, help to determine whether variations are important enough to deserve discussion.

Measures of inequality.

Managers of lawyers may want to assess the quality of a set of numbers, such as bonus distributions. Along with the well-known Gini coefficient, several other measures allow software to put a number on inequality and even pinpoint where in the set of numbers the actual data diverges from theoretical equality. These analytics help managers explain their decisions and make better decisions in the first place, if equality is sought.

Understand influence of variables and make predictions.

A whole family of regression tools goes beyond correlations. If, for example, a firm wants to predict the estimated amount of fees to be paid to it during the coming year, it could run a linear regression. The software would then point out which of the variables was more influential in predicting total fees paid and how much of the total fee paid is accounted for by the variables. One of the best-known techniques is multiple linear regression. It makes some assumptions about the relationships between whatever value is being predicted and the variables that are associated with it (e.g., level of the person retaining the firm, presence of a law department, range of practice groups involved, and years as a client).  

The regression algorithms generate a “model.” Once you have a model, you can extract information from it. A model often takes in data and makes predictions regarding new cases, clients, or matters. Think of a model as the software learning on a “training set” of data that has been labeled, such as settled for less than $10,000 or not) and applying that learning to predict something (maybe total fees) for a new case or example. With multiple regression, naïve Bayes algorithm, or neural nets prediction is a common output. For example, given a few dozen instances of a type of lawsuit, any of those machine-learning algorithms could predict the likely cost of a new matter once sufficient information is available and tell you how probable that cost would be. 

As another example of machine learning, a regression model might explain and forecast how fees and hours devoted to five common litigation tasks are associated with outcomes and therefore can predict the likely outcomes for the next case that can identify the corresponding data. Moreover, the machine-learning software can tell you which of the five tasks underpin the strongest association with the outcomes as well as how confident you can be that your prediction is correct. 

Extracting insights from text.

Words in documents can be handled statistically by software as text mining. When a survey returns free-text comments, for example, software can pick out not only which terms are used most frequently, but also assess the sentiment (the positive or negative vibes of the comments). Even more powerful are the algorithms that can assemble words from the survey comments into topics. A person has to examine the words and identify the actual topic, but the laborious work of parsing all the documents and doing the math can be done quickly by the computer. If you want to show off, mention latent Dirichlet allocation (LDA) as your topic-modeling algorithm of choice! 

A second form of machine learning would be at work when text mining software takes thousands of emails, identifies patterns in words such as repetition or proximity to each other, or pores through email messages to tag possible indicators of insider trading.

Classification and clustering.

Whenever a law firm or law department has collected a set of data, it can use a range of software tools to cluster the observations. This means that the software brings together related clients, matters, or law firms based on the information available to it about them. Once the software has clustered the observations, managers can more easily detect patterns and understand similarities and differences. A chart known as a dendrogram can depict the clustering of data and how clusters relate to each other. Somewhat similarly, software can classify observations into similar groups. Both of these types of analytics help partners see patterns that they could not otherwise detect from a massive set of data.

Other models can also classify new observations into the most appropriately fitting group. With several types of algorithms, including K-Nearest Neighbor or Support Vector Machines, you can classify clients or other data. You would be able, for example, to identify publicly-traded clients or clients likely to reach a certain realization level. 

Other varieties of machine-learning software do not require labels. Their models cluster the data into groupings that will reveal something. For example, they might cluster a firm’s clients by profitability. The K-Means algorithm can do this, and with the Principal Components Analysis you can aggregate “variables” to find out which of them is more influential.

Machine learning.

At this time, the most sophisticated data analytics that can help partners resides in a branch of artificial intelligence known as machine learning. The term encompasses a range of methods by which software chews its way through mounds of data and detects patterns. In one broad category, supervised learning, someone has to classify enough of the instances so that the computer can figure out a pattern. In another category of machine learning, unsupervised, the computer “does its own thing,” so to speak. The output can be a classification, or a regression, or other kinds of results. These tools include neural nets, support vector machines, deep learning, and Bayesian tools, among many others. This field is currently a hot spring of innovation.

4.   What Do You Need to Do to Incorporate Data Analytics More into Your Deliberations?

A.       Your firm needs a partner who is influential and exudes enthusiasm to push the initiative.  Ideally, the champion will proselytize for data analytics and secure funding. Sad, but true; you will need to ante up to find out whether and how your firm can take advantage of machine learning. 

The champion ought to be persuasive, eager to learn something about new computational tools, and adept at conveying a vision of how the firm should take advantage of the evolving capabilities of data analytics for legal management.

The champion will need to handle objections skillfully. Data can actually be feared as conspiring against the humanistic values of the partnership. Many partners in law firms shy away from data analytics because the findings invite divisive comparisons. All data discriminates. Moreover, many partners don’t really want their clients thinking about performance metrics and costs.

At this early stage of law firms exploring predictive analytics, it is very important for someone influential to explain what the benefits are and how the firm can achieve those benefits. The domain of data, software, statistics, programming, and algorithms will be mostly unfamiliar within your firm, and explanations will be welcome. A partners’ off-site conference is a good opportunity to raise awareness and attract supporters.

If IT, a practice group, HR, marketing, and a champion all have roles in a machine learning initiative, it will likely either bog down or take far too much time and money. Each group has a different interest. Someone needs to coordinate meetings, decisions, and timelines.  That project management role might fall to a junior person, or the champion might take it on.

       B.      Programming and IT Support. 

Your firm will also need programming, perhaps from a consultant or an employee.  Programmers and consultants aren’t cheap, but they are crucial. Also crucial is that any coding be work for hire, heavily commented so that someone else can follow the steps and logic, and adhere to the tenets of reproducible research.

People who have not written code for a computer to run probably don’t realize how difficult it is to code well. It is challenging to get a computer to do what you want it to do. This hurdle becomes greater as the sophistication of the programming increases, and sophisticated programming is undoubtedly required to command machine-learning algorithms. 

Your firm will need to choose software that can carry out the analyses. Those algorithms exceed the capabilities of Excel, but many other choices exist. This author relies on the open-source R programming language, which has been optimized for statistical analyses and data visualization. Another open-source choice would be Python. Many commercial packages jostle in the market, including SPSS, Tableau, SAS, and Mathematica.

As with most change initiatives, your firm should start with a pilot study and learn from it before you roll out a more ambitious project. A practice group that wants to be able to predict results, costs, or duration of matters from a subset of its past matters would be a good choice. The HR group might also apply multiple regressions on data to reduce attrition or understand better who makes partner.

    C.      Subject Matter Expert.

Your firm will need a lawyer who not only supports the initiative but also qualifies as a “subject matter expert.” A SME can look at the data set and understand the relative importance of pieces of it, what’s missing or odd, and what the firm might learn from it. A SME can translate in-the-trenches reality to the champion and programmer. For instance, looking at a set of information about certain kinds of cases, a subject matter expert could point out that the tenure of the judge — senior, mid-career, newly appointed — seems likely to correlate with the decision. An SME might also say that the duration of a case is not particularly useful because there are long stretches where neither party takes any actions. Even more usefully, an SME could classify matters as successful, unclear, or unsuccessful so that the software can tease out patterns and influential variables. 

Appendix – Source articles

Portions of this chapter came from the articles cited below, albeit with significant re-arrangement and revisions.

Rees W. Morrison, Mind the Machines: Time to Explore the Potential of Machine Learning, InsideCounsel (Oct. 21, 2016).

Rees W. Morrison, The Math Behind AI, as Explained to Lawyers, InsideCounsel (Dec. 26, 2016).

Rees W. Morrison, Drawing ACES, LegalTech News, L12 (Feb. 2017).

Rees W. Morrison, Making the Machine-Learning Switch, 25 Met. Corp. Counsel 31 (Feb. 29, 2017).

Rees W. Morrison, With Data Analytics, It's Not Always ‘Follow the Money!’, LegalTech News (March 2017).                           

Rees W. Morrison, Fairness Calculations: Letting the Gini out of the Lamp, LegalTech News (Sept. 28, 2017).

Rees W. Morrison, The Power of LDA Algorithms and How They Help Text Mine Your Documents, LegalTech News (June 8, 2017). [Text mining]

Rees W. Morrison, ANOVA Apart: How to Tell If Your Firm Averages are Actually Significant, LegalTech News (Aug. 10, 2017). [ANOVA)