Infidel, Inc.

Context and Community for Information Security

Stacks Image 233

A Guide to Forensic Testimony

The Art and Practice of Presenting Testimony as an Expert Technical Witness

Published by Addison Wesley

From the Preface:
A Bird's Eye View of This Book
"An expert is just some guy from out of town."
— Mark Twain

As usual, Twain is on the mark in suggesting that there should be something suspicious about a stranger who shows up and offers to help us with his expertise, and then quickly hits the road. For our purposes this apothegm may say even more about communities and the part they play today in deciding whom to trust as an expert, than it does about experts and how they worked in Twain's era.

This book is all about expert witnesses, with special attention paid to those who specialize in information technologies - the hardware, software, and data that make up computers and other digital systems used for data processing and communications. The level of technical expertise needed to deal with these systems often makes the question of assessing the expertise of a particular person daunting to all but other experts in the technical domain in question. The first chapter of this book introduces technical expert witnesses who testify in criminal and civil trials, and focuses on the communities of interest that society ultimately relies upon to certify the genuine expertise of their representatives and members in good standing.

When you begin to think about what it is that makes a particular individual an expert in the eyes of the law, and hence entitled to testify about his or her opinions in the course of litigation, you are led back to the specialized knowledge, training and experience that an organized and socially recognized community of interest creates and maintains. The most peculiar thing about the technical domains that comprise what is generally described as information technology is how little they resemble the traditional, professional, licensed communities of interest that exist in other areas, such as structural engineering or medicine. These communities become important to the law as it tests the reliability of the expert and his or her methods. Most judges and jurors first hear about them when a member of one of those communities is proffered as an expert witness in the course of litigation. The lack of an organized, licensed community of interest, with the traditional trappings of a socially recognized expert community creates a number of issues for IT expert witnesses with which the courts are just beginning to deal.

The authors introduce established experts from a number of communities of interest that lie outside of information technology... These experts, within ancient areas of expertise as well as brand new disciplines and sub-disciplines have coped with the special demands of the legal system. Their stories may provide some organizing analogies for IT professionals who become interested in forensic practices and enable IT experts to build the lattice of disciplines, processes, and professional networks necessary to assure lawyers and courts that they are competent IT practitioners. The experiences of Raemarie Schmidt and her students bring us back to how some of the pioneers in IT forensics can contribute to recognized expert communities by developing standards and training that have become generally recognized by the courts.

Finally, a lighthearted account of the problems that a technical expert encounters in testifying in court is offered through the discussion of the film, My Cousin Vinny. In the film, the community of expertise that is represented by the character, Mona Lisa Vito (played to perfection by Marissa Tomei) is that of the automobile mechanic. This particular community reminds us that certain roles associated with IT are rapidly becoming as commonly accepted as those of the car mechanic and the stove or furnace repairperson. That these areas of expertise are generally recognized and often celebrated raises another specter for the community of interest. In this scenario, too many members claim expertise with too little self-regulation, peer review and evaluation by a recognized community of professionals. This erodes the ability to separate the charlatans from the qualified and recognized practitioners of the information technology trades.

Chapter 2 provides a real world tale of just how serious this kind of communication performance can be to individual and corporate parties. This chapter also explores the kind of expectations that legal and IT social critiques bring to bear on these expert performances by important IT witnesses in landmark cases. Selecting passages from the deposition of Bill Gates in the Microsoft antitrust case, a number of the recurring themes and issues associated with expert testimony that will be developed further in the rest of the book are highlighted and introduced. The most important of these is the perception of the demeanor and overall credibility of the witness and his performance on the stand. This perception by the fact finder overrides, as it should, all of the other components of the process of communicating complex concepts in formal testimony as a witness under oath. The return of Bill Gates to the witness stand two years later, and the dramatic change in the reporting of his second coming by the same IT and legal reporters is in and of itself the proof of the pudding for this book. Judicial fact finders and the public have lofty expectations of the expert witness, especially when the expert's testimony is key to understanding the merits of the case. Meeting those expectations requires certain things from the expert: experience, preparation and a commitment to communicating not only the obvious expertise of the witness, but also the credibility and willingness to provide useful information throughout the performance of testimony. This set of requirements might appear excessive, but in certain cases, such as Mr. Gates', the members of the public with interest in the expert's testimony number in the millions.

Chapter 3 reprises the well known story of how IT security experts Tsutomu Shimomura and Andrew Gross developed forensic tools to track down the hacker who broke into Shimomura's computer at the San Diego Supercomputing Center. The investigation is recounted in the form of a hypothetical direct examination of Andrew Gross as the government's expert witness and illustrated with graphics, designed to be used to introduce, illustrate and argue the complex technical steps that were taken in the investigation. The testimony also explains the expert analysis of the computer network evidence used to establish that Kevin Mitnick was the original intruder or at any rate to account for how he came to possess the stolen computer data taken from Shimomura's computer.

Chapter 4 provides some historical background for the reader. It outlines the evolution of the legal process and also explores the growing importance of expert witness testimony that accompanies the evolution of society's dependence on technology. The different roles of the expert witness, as consultant, strategist and testifying witness are introduced along with some of the problems that can arise when these often-conflicting roles are not kept clear and distinct by the expert and his or her attorney throughout the course of litigation.

Chapter 5 gives the beginning expert several examples by analogy of the kinds of problems that may persist, due to the pace of advance in information technology. Some of the problems are considered to be a direct consequence of the inherent immaturity of much of information technology. In particular, there are issues arising in areas where a scientific or rigorous community of interest has not yet been established, or where no formal educational or training is available... In these cases, the expert cannot point to generally accepted standards or a formal peer review process for determining the reliability of the concepts and techniques that he or she uses to decide what happened in a given case. Astrologers, phrenologists, handwriting and fingerprint comparison experts and their communities of interest are discussed to illustrate the kinds of problems that may be encountered by a number of IT domain experts, when their expertise is challenged in court.

Chapter 6 provides examples, many of them extreme, of what can go wrong when commonsense rules of professionalism and ethics are misapplied. It also outlines how the traditions of the legal system regarding the preparation and introduction of expert testimony place certain restrictions on the behavior of IT and other experts who are performing expert witness tasks in the course of litigation. Above all, the expert must understand that in civil litigation, the expert is ultimately working for a private party, through their legal counsel, but that the understandably biased advocacy decisions that are made by the party and the attorney about the course of litigation, must be segregated from the sound, objective judgments that the law and professional ethical rules require an expert witness to make about the application of their expertise and the communication of their qualified opinions to the court and the jury.

Chapter 7 shows how some experienced IT experts have handled the challenging task of ensuring that the professional relationship that must be established between the expert and the attorney and party to the litigation is solidly and clearly constructed and maintained. One of many useful metaphors for enabling both the lawyer and the expert to reach useful conclusions about issues within the expertise of the witness is to think of the formation of this relationship as a checkout for a flight. This analogy requires the expert to learn a lot about the role of being an expert before he or she is in a position to check out all the things that need to be in working order and to notice all of the indicators of problems before taking the plane into the air. Other approaches that have worked for both beginning and experienced IT expert witness practices is to find an agent or agency that specializes in matching appropriate experts with legal teams requiring particular expertise.

Chapter 8 is in some ways the most difficult material presented in the book. This is the most involved legal material offered to the reader and explores the kinds of criteria that courts have established for expert witnesses in general. The legal approach to screening expert witnesses has undergone significant change over the past decade through a series of Supreme Court decisions. The result of this series of decisions, starting with the landmark case, Daubert v Merrill Dow Pharmaceuticals is that there are additional tests that an expert must pass in order to be allowed to testify as an expert. The major differentiation between old and new processes associated with qualification of expert witnesses involves the addition of a "gatekeeper" function, assigned to trial judges.

The gate keeping function that most courts have now accepted in one form or another is a distinct departure from the traditional role of courts with regard to use of experts. Under the old system, the courts passively allowed attorneys to proffer their chosen experts, allowing the jury to decide what weight to give to the respective witnesses' opinions. In the post Daubert world, the judge acts as a "gatekeeper" and is charged with weeding out unqualified experts and qualified experts who deliver unreliable opinions that are not relevant to the particular case at hand. This function is implemented by providing a process enabling adversaries to challenge the qualifications or relevance of a particular expert. The challenges are conducted in addition to and in advance of the more traditional impeachment of witnesses through cross-examination.

This expert qualification and challenge process requires additional work on the part of expert witnesses. First, the expert needs to consider whether he is adequately qualified by education, training and experience to investigate and opine about particular matters before the court. While acting as an expert witness, he must also keep abreast of legal developments and to make additional efforts to determine all that will be involved in a particular assignment. Experts may also need to determine how a soliciting attorney has dealt with past cases before the judge in which gate keeping challenges were mounted to his or her experts. Finally, the expert must accept the additional responsibility for anticipating and dealing with serious challenges to his qualifications and expertise. Finally, this rapidly changing body of case law needs to be understood on the fly, as it has only recently been used to challenge IT expertise and experts.

The good news is that experts don't have to take this on all by themselves. All competent trial lawyers can be expected to have kept up with the most recent changes in the way this challenge round is evolving in their jurisdictions, and should be able to explain it clearly to the beginning expert. The information in this chapter is presented in hopes that it will enable IT experts to pose relevant and concise questions about this new area of the law and at the same time prepare them to better understand the significance of the legal advice they receive from trial counsel concerning these new developments and their impact on the performance of the witness.

Chapter 9 provides a detailed example of how judges look at qualifications and approach in deciding between competing theories and methods of opposing experts. The landmark case of Gates v. Bando, which established one of the practice standards for computer forensics, is examined. In particular, the testimony of Robert Wedig, the expert witness for the defendant who prevailed in that case, is assessed and the factors affecting the judge's decision to favor Dr. Wedig's opinion over that of the opposing expert articulated. In this chapter the historical example of Houdini is used to illustrate the different roles of the expert - as performer demonstrating a known expertise, and the expert taking on the different role of skeptic, uncovering the abuse of known techniques or an unknown expertise used by an opposing expert or attorney to obfuscate the facts, or to deceive the fact-finder.

One of the most important ancillary measures that can be employed by an IT expert in court is that of visual displays. Chapter 10 takes the subject of graphic images into detail and provides a visual metaphor to allow the beginning expert to think about the entire process of approaching a technical problem involved in litigation through the eyes of graphics designers. The litigation graphics consultants who work with lawyers and their technical experts enable them to focus on the most important concepts and to organize their presentations with the aid of graphical tools. The resulting visual displays vastly enhance the expert's ability to communicate the analysis and conclusions to judges and juries. Several examples of the work of one of these consultants, TheFocalPoint, are provided. In addition, the methodology that Chris Ritter, a former litigator who now works for TheFocalPoint, has developed to be able to assist both lawyers and expert witnesses to prepare for court is also included.

Although experts are often tempted to focus on the content of their testimony, in court, the context of testimony is also very important. This means that even the most brilliant and accurate technical analysis may not be accepted if the demeanor and nonverbal communications skills of the expert witness are lacking.

Chapters 11 and 12 contain various analogies and techniques for improving the ways that witness demeanor and nonverbal communication skills can be integrated with expert testimony.

With all the provisions and restrictions of the legal process outlined, Chapter 13 provides a wealth of wisdom from the front lines. This chapter concludes with the advice of several noted IT experts with differing degrees of experience. Professor Gene Spafford, noted for his accomplishments in the software engineering and network security area, offers insights gained over his decades of testifying in cases involving intellectual property theft and patent infringement. Don Allison, whose expertise in the discovery and analysis of digital evidence has placed him on the stand in the tawdriest of child pornography cases, offers his feedback loop methodology that not only carries him through the trial process, but also allows him to refine his expert witness skills. Finally, Professor Rebecca Mercuri, a world-renowned specialist in the area of voting technology, offers insights gleaned over a decade of testimony in a wide variety of cases, ranging from a murder trial to the appeal of the 2000 U.S. Presidential Elections results in Florida…

James Boyd White, a law professor and respected author has written:

Perhaps I am answering a voice, in myself or in the culture, that says that there is no such possibility; that law is only the exercise of power by one person or group over another, or only a branch of bureaucracy, or only moneymaking, or only instrumental; that it has no real and independent value for the person or the community. Thus I ask whether we can imagine law as an activity that in its ideal form, at least on occasions, has true intellectual, imaginative, ethical, and political worth.

In doing this I try to show that the law can be seen as a particular instance of a human activity that is far more widespread than law itself, and of which we have splendid exemplars from which to learn: the activity of making meaning in language in relation to others. To see law this way opens a whole set of issues for analysis in the law (and in other instances of meaning-making too): the quality of the language that a particular person inherits and uses; the nature of her transformation of that language in her use of it; and the kind of relation she establishes with the people she speaks to or about.

In several thousand times as many words, this book attempts to explain the significance of Mark Twain's single, cynical quote and to do it with the optimism and hope of Professor White.

At the outset, it should be made clear that the authors, unlike Samuel Clemens, are not professional writers, humorists, or entertainers extraordinaire. If they could have figured out how to make a living being professional philosophers, one can assume that they would have done so already. Once it was clear that they were unable to make their livings writing, entertaining, or philosophizing, they chose instead to pursue careers in the fields of information technology and law, respectively.

Nor, in retrospect, are the authors as clear as they would like to be about the points they set out to make in all these pages. The problem that brought the authors together to write this book emerged from doubts about the ability to communicate the idealistic goals tempered by the cynical observations that color the world of the expert witness. They concluded that simply sending something as wonderful as the Twain quip, and as thoughtful as the White quote, together, to all the potential technical experts who had asked them for advice would be viewed as inadequate. Even assuming that their inquirers would like the philosophical position of Professor White when pondering the problem of testifying as an expert witness in the real world (given the way the real world in Twain's view really is) they would still need a path to follow.

So, the main impetus for writing this book was to define some paths that could allow technical experts to more easily gain an understanding of why it is critical for those qualified and capable of joining the fray, to do so. For in doing so, they contribute to Professor White's "making of meaning in language in relation to others" involved in the litigation of information technology issues.

Our subject then, is how to master the art of presenting effective information technology expert witness testimony. This testimony, in the best case, enables a judge or a juror to make meaning in relation to complex technical concepts involved with information technologies. This art in turn enables the fact finders in litigation to relate that meaning to an important controversy, in order to make sound judgments about it.

And yet, Twain must keep bringing us back with his ten words to the way we suspect things really are in the world of litigation. And we realize that we must deal with that as well. No one wants to be perceived as a circus clown in a setting where everyone else is pretending to be serious about another game. Our suspicion that we are being foolish or that we will be made the fool, by following Professor White in his optimism about the law and our legal rituals, makes it all the more difficult to sustain such optimism. Yet the sustained effort to communicate carefully and objectively the professional experiences and special knowledge we wish to share with others is not difficult to justify. It is both the essence of the scientific method and the most rewarding kind of trial advocacy.

We have chosen to undertake the job of overcoming the cynical force of Twain's apothegm with analogies, metaphors, stories, disciplines, opinions and anecdotes. It is our hope that the humor of some of these stories and the insights and wisdom of some of the opinions and analysis of other experts will go some way in overcoming that convenient cynicism. There will always be those who would rather remain uninvolved, while technical experts and their lawyers (from out of town) increasingly command the center stage of a growing number of legal performances. Our hope is that their numbers will be reduced by some who are challenged by the materials collected in this primer.