18. The Problem of Error

If nobody benefits by enforcing a legal rule, the rule will not be enforced. If someone benefits too much by convicting someone else of violating a legal rule, we may spend more on law enforcement than the resulting improvement is worth. Getting the incentive correct involves balancing the benefits of enforcing the law against the costs. We could get almost perfect enforcement of speed limits if every road had a cop car every few miles, but it would take a lot of cops and cars.

A second cost of convicting more who are guilty is convicting more who are not. If we increase convictions by reducing the amount of evidence required to convict, we will acquit fewer guilty defendants but convict more innocent ones. If we increase convictions by increasing the reward to cops who arrest speeders, we have increased the incentive for cops to arrest drivers whom they falsely claim were speeding and for localities to contrive speed traps designed not to reduce accidents but to increase revenues. It is a problem almost as old as automobiles.

While we waited outside, the fat man on the grey horse rode up and entered into loud talk with his brother magistrates. He said to one of them—for I took the trouble to note it down—‘It falls away from my lodge-gates, dead straight, three-quarters of a mile. I’d defy any one to resist it. We rooked seventy pounds out of ’em last month. No car can resist the temptation. You ought to have one your side the county, Mike. They simply can’t resist it.’

Kipling 1918, 191

A different example of the same problem was described in Chapter 15. In England in the mid eighteenth century, both Parliament and the Crown established rewards for convicting criminals of offenses for which it was feared that private prosecution was inadequate. The result was a series of scandals in which someone was either entrapped into committing an offense or framed for an offense he did not commit. Eventually rewards for conviction were replaced by partial compensation for the expenses of a successful prosecution—not enough to make prosecution profitable, merely less costly.

Preventing False Verdicts

The direct approach is to use legal rules that make it hard to convict innocent defendants. The rules of evidence and the requirement of proof beyond a reasonable doubt in Anglo-American criminal law are supposed to achieve that. It is not clear how well they succeed in a legal system where the overwhelming majority of convictions are not by trial but by plea bargaining. We have no good measure of what fraction of those convicted of crimes are innocent but attempts at indirect measures for serious offenses suggest that it is at least three to five percent, perhaps more. Given the logic of plea bargaining, the rates for less serious offenses might be considerably higher.

Even if we knew how to design rules that did a better job of separating innocent from guilty, it is unclear that it is in the interest of those who create the rules to do so. When someone is convicted of a crime he did not commit it is not the judge or the legislator who pays the penalty. And even with well-designed legal rules it may often be possible to find, out of the universe of all potential suspects, someone easier to convict than the actual offender. Hence, even with well-designed rules, increasing the incentive to convict can be expected to increase the number of false convictions.
The problem is the same whether the incentive is a reward for catching a criminal, avoiding the punishment for failing to catch a criminal, tort damages, career advancement for a police officer who gets his man or favorable publicity for a politically ambitious prosecutor. In modern American law the issue of too much incentive to convict shows up as concerns with civil forfeiture, class actions and punitive damages. In feud law, the corresponding problem is extortion disguised as rights enforcement.

Blackstone famously wrote “Better that ten guilty persons escape than that one innocent suffer.” Benjamin Franklin made it a hundred to one. Neither offered any basis for their number.

If only we had a legal system that would never convict an innocent defendant…

The Origin of the Law of Torture: A Cautionary Tale

People in the past worried about convicting the innocent too. In the early Middle Ages, they had a solution—let God judge. A defendant could be subjected to an ordeal such as plunging his hand into boiling water, carrying a red-hot iron bar, being dumped bound into water. Various passages in the Bible were interpreted to imply that God would reveal guilt (hand injured or body floated) or innocence (not injured, sank, pulled out). Since God was omniscient, a correct verdict was guaranteed.

The use of ordeals was eventually abandoned on theological grounds; in 1215, the Fourth Lateran Council rejected the religious legitimacy of judicial ordeals and banned priests from participating in them. Over the next few decades most European countries abandoned their use.

That left medieval judicial systems looking for another way of being certain a defendant was guilty before convicting him. The solution was a high standard of proof, evidence “clear as the noonday sun.” Conviction required either two unimpeachable eyewitnesses to the crime or a voluntary confession. Circumstantial evidence, however strong, was insufficient.

In the history of Western culture no legal system has ever made a more valiant effort to perfect its safeguards and thereby to exclude completely the possibility of mistaken conviction. But the Europeans learned in due course the inevitable lesson. They had set the level of safeguard too high. They had constructed a system of proof that could as a practical matter be effective only in cases involving overt crime or repentant criminals. Because society cannot long tolerate a legal system that lacks the capacity to convict unrepentant persons who commit clandestine crimes, something had to be done …

Langbein 1978, 4-5

The solution was the law of torture. Once the court had half-proof, one eyewitness or the equivalent in circumstantial evidence, the defendant could be tortured. A confession under torture was not voluntary, but that problem could be dealt with. Stop the torture and the next day ask the defendant if he is willing to confess. Since he is now not being tortured, the confession is voluntary. If he doesn’t confess, torture him again.

John Langbein, my source for this account, offers a parallel story in modern law. Two hundred years ago, jury trials were short:

In the Old Bailey in the 1730s we know that the court routinely processed between twelve and twenty jury trials for felony in a single day. A single jury would be impaneled and would hear evidence in numerous unrelated cases before retiring to formulate verdicts in all. Lawyers were not employed in the conduct of ordinary criminal trials, either for the prosecution or the defense. The trial judge called the witnesses (whom the local justice of the peace had bound over to appear), and the proceeding transpired as a relatively unstructured “altercation” between the witnesses and the accused. In the 1790s, when the Americans were constitutionalizing English jury trial, it was still rapid and efficient. “The trial of Hardy for high treason in 1794 was the first that ever lasted more than one day, and the court seriously considered whether it had any power to adjourn… .”

Over the years since, trials have become longer and much more complicated, at least in part in an attempt to reduce the risk of convicting the wrong person. Patricia Hearst’s 1976 trial for bank robbery lasted forty days. That was unusually long, but the average felony jury trial in Los Angeles in 1968 took 7.2 days, more than a hundred times the length of the average felony trial in the Old Bailey in the 1730’s. If every felony trial in the U.S. took that long, felony trials alone would require the full-time efforts of more judges than currently exist, state and federal combined, and close to a million jurors, court attendants, and the like.608 Not impossible, but very expensive.

The American legal system found a cheaper alternative. Like its medieval predecessor, it substituted confession for trial. The medieval confession was motivated by the threat of torture. The modern version, a plea bargain, is motivated by the threat of a more severe sentence if the defendant insists on a trial and is convicted. Like the medieval version, it preserves the form—every felony defendant has the right to a jury trial, a lawyer, and all the paraphernalia of the modern law of criminal defense—while abandoning the substance. Conviction after a lengthy and careful jury trial is, arguably, evidence of guilt beyond a reasonable doubt. The willingness to accept a sentence of a year, possibly a year already served while awaiting trial, instead of the risk of ten years if convicted is not.

Similar problems have arisen in other legal systems. Under Jewish law as described by Maimonides, conviction for a capital offense required two witnesses to testify that, immediately before the offender committed the crime, both of them warned him that it was subject to capital punishment and he committed it anyway. And the witnesses must have seen the crime committed:

“Even if the witnesses saw him (the assailant) chasing the other, gave him warning, and then lost sight of him, or they followed him into a ruin and found the victim writhing (in death agony), while the sword dripping with blood was in the hands of the slayer, the court does not condemn the accused to death, since the witnesses did not see him at the time of the slaying.”

That was the theory. The practice:

“Every court … if it sees that crime is rampant among the people … can impose the death penalty, monetary fines, or other punishments, even if there is no absolute proof.”

Under Islamic law, conviction for a Hadd offense, one based in theory directly on the Koran, required two eye witnesses to the same act, both competent adult Muslim males of good reputation. In the case of Zina, unlawful intercourse, it required four witnesses to the same act of intercourse, not a condition likely to be met. Hadd crimes could, however, be prosecuted as Ta’zir offenses, with weaker standards of proof and, according to some but not all schools of law, weaker punishments. If even that was too difficult, there were the shurta and mazalim courts set up by the ruler and not bound by the rules of Koranic jurisprudence.

Standards of Proof

In setting the standard of proof, a legal system is balancing the benefit of convicting more guilty defendants against the cost of convicting more innocent defendants. Conviction of the innocent may be less important or deterrence more important for some offenses than for others. That would be a reason to use different standards of proof for different offenses.

In Anglo-American common law, the standard of proof for a civil case is a preponderance of the evidence, commonly interpreted as a probability of guilt above fifty percent. The standard in a criminal case is proof beyond a reasonable doubt. Why the difference? It cannot be just a matter of higher standards when more is at stake, since the criminal standard applies to minor crimes as well as major, the civil standard to suits claiming tens of millions of dollars in damages as well as suits for small amounts.

One possible answer is that a mistake in the civil case means that someone pays money that someone else gets. That may be unjust, but the cost of the penalty nets to zero. In the criminal case, nobody gets the life or the liberty that the innocent defendant loses. So convicting the innocent is more costly in the criminal case, which is a reason for a higher standard of proof.

The argument can be generalized both to standards of proof in other legal systems and to other ways of holding down punishment costs. In Islamic law, giving the victim or his heir the choice between retaliation and diya, blood money, makes it possible to substitute an efficient punishment for an inefficient one. So did out-of-court settlements in English criminal law, legal and encouraged for misdemeanors, illegal but, I have argued, common for felonies.

A legal system could compensate for a higher standard of proof with more severe punishment of those convicted, thus maintaining the same level of deterrence. One cost of that approach is that it may require a shift from an efficient punishment, a fine, to less efficient alternatives such as imprisonment, corporal punishment, or execution, there being a limit to how large a fine a defendant can pay.

Technologies for Revealing Truth

One way of avoiding the conviction of innocents is to require a high standard of proof. Another is to change the rules in ways designed to do a better job of judging guilt. That might mean longer trials, greater expenditure by the court and the parties on proving or disproving charges. It also might mean changes in how cases are tried. What changes improve performance, what changes worsen it, may depend on details of the legal system and the society.

Consider jury selection. Early medieval England used an informed jury, jurors local to the defendant and so likely to already know much of the information relevant to his guilt or innocence. That advantage was balanced by the risk that they might be biased for or against a defendant they knew. The modern U.S. system takes the opposite approach, doing its best to exclude from the jury anyone with any connection to the defendant and relying on the trial process to inform an initially uninformed jury. Which approach is better depends in part on which is more likely to be biased, the neighbors or the trial apparatus, in part on how rich the society is. A poor society may be better off relying on the information the jurors already have instead of throwing that information away and paying someone to replace it.

Another issue is the choice between an inquisitorial system, in which the court apparatus discovers and presents evidence to judge or jury, and an adversarial system, where the prosecution offers evidence for guilt, the defense for innocence. A disadvantage of the latter is that someone looking for evidence on one side may ignore any he turns up on the other. An advantage is that, with a partisan on each side, there is less risk that partisanship on the part of the investigator will produce a biased selection of evidence.

There is another and less obvious advantage to the adversarial system. The payoff to looking for something is higher the more likely you are to find it. An innocent defendant can expect that the facts will, on average, support his innocence, and so has more incentive to pay the costs of an extended search for evidence than a guilty defendant. Thus the self-interested behavior of the defendant reveals his private information of guilt or innocence in a form, evidence of innocence produced, that feeds into the trial process. The inquisitorial system has no such mechanism, since the search for evidence is being done by court officials with no access to the defendant’s private information of guilt or innocence.

An argument on the other side is that, under the adversarial system, how much is spent looking for evidence of innocence depends in part on how rich the defendant is. The inquisitorial system can choose to spend the same amount investigating the innocence of rich and poor.

Another way that the accuracy of trial outcomes might be improved is by reducing restrictions on how evidence can be obtained. A modern example is surveillance. Tapping phones, setting up video cameras on poles in public places and recording their output, requiring phone companies to keep records of who called whom when and make them available to law enforcement, monitoring activities on the internet, searching homes and computers, are all technologies that can be used to gather evidence to help identify the guilty. Such technologies also have costs: They reduce privacy and provide opportunities for blackmail by law enforcement agents, agencies or the politicians they serve. Those are reasons to restrict such activities.

One approach to doing so in modern U.S. law is the exclusionary rule: Evidence obtained by an illegal search cannot be used at trial. That reduces but does not eliminate the incentive to conduct an illegal search; the information produced might be used without its origin being revealed. According to news stories published in 2013, evidence produced by surveillance justified for national security was being funneled to law enforcement agencies acting against domestic crime. The recipients were instructed that, once they had the data, they should reconstruct it by normal investigatory techniques and, at trial, conceal its origin.
I have already mentioned an older investigatory technology still not entirely abandoned: torture. One argument against it is that a defendant who is tortured and acquitted has been punished by torture, perhaps permanently injured, for a crime he did not commit. Another, noted at least as far back as Periclean Athens, is that under torture even an innocent defendant may confess.

The law of the Visigoths, the earliest Germanic law code to have survived, offered a solution to the second problem. Torture of a defendant was only permitted if there were facts of the crime that an innocent defendant would not know. A confession was only accepted if it included such facts. The same approach is sometimes used in modern law in deciding whether a confession should be believed.

Both the ancient and the modern rule have the same weakness. The interrogator knows the relevant facts. If he wishes to convict the defendant, all he has to do is leak the facts to him when nobody else is around and then use torture to compel him to include them in his confession.

Suppose there were a substitute without those problems, a truth drug that produced reliable information with no harm to the defendant. The arguments against torture would then not apply. Should we use it?
A possible response is that we want to do a good job of enforcing the law, but not too good. So far I have taken it for granted that the laws being enforced are ones that should be enforced. That may not always be the case. A reliable way of convicting all criminals is also a way of enforcing tyrannical laws. Limits to law enforcement technologies may be justified as limits on the power of whoever is making and enforcing the laws.

Changing Incentives

Another approach to the problem of protecting the innocent from conviction is to make it in the interest of prosecutors to avoid prosecuting innocent defendants. An example is provided by Athenian law. Their equivalent of criminal cases could, like criminal cases in eighteenth-century England, be privately prosecuted by any adult male (in the English case also female) citizen. Conviction usually led to a large fine, a share of which went to the successful prosecutor. To reduce the risk that a prosecutor might deliberately target an innocent defendant, Athenian law provided that a prosecutor who failed to get at least 20% of the jury to vote for conviction was himself fined and barred from any future prosecution of that sort of case. That gave potential prosecutors an incentive to avoid prosecuting defendants who had a substantial chance of proving their innocence. In the Guta Lag, a thirteenth-century Swedish law code, a woman found guilty of killing her child owed a three mark fine. If she was found innocent, her accusers owed her three marks.

The Athenian approach could be applied to a modern legal system by penalizing a tort plaintiff or public prosecutor who failed to get more than some minimum number of jurors to vote for conviction. Alternatively, one might impose a penalty on prosecutors responsible for convictions later shown to be mistaken, as in the case of DNA reversals.

The problem with the first version is that it gives prosecutors who suspect the defendant may be innocent an incentive to conceal as much as possible of the evidence from the jury. The problem with the second is that it gives a prosecutor who suspects that he may have convicted an innocent defendant an incentive to prevent the error from being discovered.

That is not a purely theoretical issue. There is no legal penalty in modern U.S. law for a prosecutor who turns out to have convicted an innocent defendant but there are costs to his reputation. Reading accounts of attempts by innocence projects to use DNA testing to check the results of old cases, one is struck by how reluctant many of the authorities controlling access to the evidence were to make it available.

For a different approach to protecting the innocent, consider prosecution motivated by private deterrence. Arresting the first feeble-minded beggar found near the site of a crime and railroading him to the gallows works just as well as hunting down the actual perpetrator from the standpoint of a private prosecutor motivated by a reward or a police officer angling for promotion—and it may be a lot less work. But if potential felons know whether the person convicted is the one who committed the crime, as the real criminal certainly does, private deterrence depends on convicting the right person. It is thus one mechanism for prosecution, perhaps the only one, that comes with a built-in incentive to convict the right person.

Enforcers are not the only people whose incentives matter. Criminals and witnesses matter too. In many legal systems punishment is reduced for an offender who turns himself in or increased for an offender who takes actions, such as concealing the body of his victim, that make it harder to discover the crime and assign blame. That too is a mechanism for making it easier to selectively convict the guilty. In many legal systems, perjured testimony is itself criminal, making it less likely that innocent defendants will be convicted or guilty acquitted. In several, including Jewish law and the legal system of Periclean Athens, false testimony that led to the execution of an innocent defendant was a capital offense.

Legal Systems and the Incentives They Provide

So far we have been looking at the problem of the incentive to enforce and the associated problem of the conviction of innocent defendants from a perspective that cuts across legal systems. How does the logic apply to differing mechanisms of enforcement?


Someone deciding whether to demand compensation for injury has two incentives do so, the compensation and deterrence. He has one incentive not to—the violence that may result, including both the immediate clash if he tries to carry out his threat and subsequent conflicts if the other party retaliates. The clearer it is that he is in the right and the other in the wrong, as judged by potential allies of both, the more likely the other is to give in to his demand and the lower the expected costs of conflict. How likely a pure feud system is to punish the right people thus depends on how accurately others in the community can judge guilt, which explains why such systems typically include some mechanism for third-party arbitration. The verdict of a trusted arbitrator or court provides interested third parties with information on who is at fault without requiring them to investigate the claims of the parties themselves.

The same considerations come in on the benefit side of the calculation. Starting a feud only provides deterrence against future wrongs if those who might commit them believe that you really have been wronged, hence take your response as evidence that it would be prudent for them not to wrong you. The less clear the evidence for your position is, the less benefit you get from demanding compensation or, if it is not forthcoming, carrying out the threatened violence. If your neighbors consider your action entirely unjustified they may conclude not that it is risky to wrong you but that it is risky to have anything to do with you, perhaps even that you are a public danger who should be dealt with.

Reputational Enforcement

Tort law, criminal law and feud deter offenses by imposing penalties on offenders; the offender is punished due to actions by someone trying to punish him. Reputational enforcement also imposes costs on an offender, but those costs are an indirect effect of individuals choosing to protect themselves by not dealing with parties they believe cannot be trusted.

If their judgment is correct, their action benefits them; if they are wrong, they are giving up opportunities for potentially beneficial transactions. That gives them an incentive to distinguish innocent from guilty and, unlike prosecutors who will be rewarded for a conviction, no incentive to convict the innocent. But while there is no benefit to convicting the innocent there is in most cases only a small cost to doing so, since there are usually other department stores and other diamond merchants to deal with. Hence the third parties whose acts provide reputational enforcement have only a weak incentive to avoid convicting the innocent.

How serious this problem is depends on how easily third parties can determine who is at fault in a dispute. If doing so is hard and the benefit small they may instead assign some probability of guilt to the offender, some to the complaining victim, and avoid dealing with both—in which case, as pointed out in the previous chapter, complaining becomes a losing strategy and reputational enforcement breaks down. That is again an argument for mechanisms, such as arbitration, that provide third parties with information without requiring them to independently judge the merits of a controversy.


Punishing rules violators by ostracism has one attractive feature in common with reputational enforcement: The punishment is costly for the ones who impose it as well as the one it is imposed on, giving the former some incentive to get it right. On the other hand, it is easy to imagine circumstances where one member of a community can benefit by expelling another, perhaps a rival in love or communal politics. If the former is influential or very persuasive and the latter unpopular, there is an incentive for, and risk of, false conviction.

Divine Enforcement

If potential offenders believe in a supernatural power that punishes those who violate divine law, they have an incentive not to do so. Religion provides the perfect mechanism for punishing the guilty and only the guilty, provided that the religion is true and the divinity omniscient. It works reasonably well even if the religion is not true, provided that potential offenders believe in it. A traditional Romani has an incentive to avoid pollution even if he is sure nobody is watching, although less than when supernatural sanctions are reinforced by social sanctions, and similarly, mutatis mutandis, for a believing Jew or Muslim.

There are problems with this approach if implemented without the assistance of an actual divinity. Religious law must be interpreted by someone. If the interpretation is by the individual believer, he may commit actions he ought not to if he can persuade himself that what he wants to do is not sinful—and every man is a biased judge in his own case. If interpretation is by others, they may use the threat of divine punishment to serve mundane objectives.

Peter Leeson, in an article on medieval ordeals,616 describes an ingenious mechanism to leverage religious belief in the service of human enforcement. The legal system was structured in a way that gave accused defendants the opportunity to undergo an ordeal or avoid one. Since defendants believed that God would reveal guilt or innocence, guilty defendants were reluctant to undergo ordeals. The priests, realizing that most of those who chose ordeals were innocent, rigged the ordeals to acquit most defendants. In support of that interpretation, Leeson offers evidence of an implausibly large number of accused criminals who succeeded in picking up purportedly red-hot iron or plunging their hands into boiling water without injury. He also points to a collection of cases where the ordeal of submersion was given only to men—because, he argued, women were more likely to float and so be convicted.

Correcting Mistakes

A final issue is what can be done to correct mistakes when and if they are discovered. The answer depends in part on the form of punishment. If it is a fine or a damage payment the money can, at least in principle, be returned. If it is a continuing punishment, such as imprisonment or ostracism, the convicted innocent can at least be relieved from the remainder of his sentence. When it is discovered that someone who was punished was innocent he can be compensated with money, provided he is still alive, and some modern legal systems attempt to do so. If a defendant is discovered to be innocent only after he has been executed, on the other hand, there is not much that can be done to compensate him.

Another possibility, if a verdict is shown to have been mistaken, is to punish those responsible for the error. If the false conviction is due to misdeeds by the legal apparatus, a prosecutor who withheld exculpatory evidence or a police officer who gave false testimony, it seems just to punish those responsible. The problem with this approach, as mentioned earlier, is that establishing the innocence of someone who has been convicted of a crime he did not commit is likely to require the cooperation of the legal system that convicted him, which is less likely to be available the greater the risk to members of that legal system.

Добавить комментарий

Ваш адрес email не будет опубликован. Обязательные поля помечены *