Credit: Getty Images

American crime rates, especially violent crime rates, and American incarceration rates are twin national disgraces. We have five times the homicide rate and five times the incarceration rate of other economically advanced countries. Both crime and incarceration are appallingly concentrated among poor African Americans; in the same neighborhoods where homicide is the leading cause of death for young men, more than half of those men will do prison time before they turn thirty.

The concentration of incarceration by race is by now a well-worn topic. Some activists and scholars allege a concerted effort to replace older forms of racial oppression with the penitentiary. The concentration of incarceration by social class is less well known, but no less worrisome.

What that critique leaves out is the concentration of crime. Violent crime has fallen 67 percent from its peak in the early 1980s and early ’90s, but remains more than twice as common as it was before the great crime wave of the ’60s. And crime is just as concentrated as incarceration: blacks are about six times as likely as whites to be imprisoned, and also about six times as likely to be murdered. Almost all of those homicides are intraracial. The Crips and the Bloods killed more African Americans in the last quarter of the twentieth century than the Ku Klux Klan killed in its entire history. Homicide rates have fallen sharply over the past two decades, but that may have more to do with improved shock-trauma medicine than with reduced criminality; the rate of gunshot wounds has not fallen.

The actual bloodshed may not be the worst of it. The costs of crime are both enormous and underappreciated, because they consist primarily not of the direct losses to victims of crimes but of the costs people and businesses incur, and inflict on one another, in attempting to avoid victimization. Every store that moves away from a poor neighborhood for fear of robbery takes with it both services and jobs, leaving the neighborhood that much poorer and more socially isolated.

Crimes against African American victims and in poor African American neighborhoods tend to be underpunished. Historically, that reflected the broader society’s indifference to the lives and property of the descendants of slaves. Police on the beat today can still remember when their superiors on the force dismissed black-on-black killings as “NHI”—”no humans involved.” The continuation of underpunishment today reflects a variety of causes: the political weakness of poor black neighborhoods; the distrust between residents of those neighborhoods and the police, leading to “no-witnesses” killings in broad daylight; the sheer volume of cases in crowded urban courts, leading to cheap plea bargains; and the reluctance of prosecutors and judges to contribute further to disproportionate incarceration.

But whatever the causes, the result—as Randall Kennedy pointed out many years ago—is to deny poor African Americans the “equal protection of the laws” commanded by the Fourteenth Amendment. Their persons and goods are less safe from criminal victimization than those of richer people with lighter-colored skins. This is as serious a denial of social justice as the better-publicized—and shrinking—black-white educational gap.

It is also a self-reinforcing social trap. Growing up in a high-crime neighborhood is a major risk factor for becoming criminally active oneself. Living in such a neighborhood reduces the benefits of lawful activity by reducing economic opportunity and making lawfully acquired property less secure. It also reduces the costs of unlawful activity by surrounding a potential offender with people who are easy to victimize and either not inclined or not able to command the attention of the authorities. Growing up in such a neighborhood also means a higher likelihood of being a crime victim or witnessing violence, two more factors that make someone more likely to offend.

Moreover, a generally chaotic environment tends to reduce the ability of children who grow up in it to control their impulses in the service of long-term goals. And, as Glenn Loury notes in these pages (“Prison’s Dilemma,” page 46), the threat of a prison term tends to have less deterrent value in a high-crime neighborhood, because going to prison counts as a routine life incident rather than a shocking personal and family disgrace. The result is that underpunishment and overincarceration often coexist, working their joint devastation from generation to generation.

Is there a way out of this social trap? The conservative project of moral denunciation and harsh punishment, even if it were morally sustainable, has clearly passed the point of diminishing returns. The standard progressive approach of social services (such as drug courts, reentry programs, and “justice reinvestment”) has great surface appeal but little in the way of evidence of efficacy. Could reduced racial discrimination and greatly expended legitimate economic opportunity make a big difference? Perhaps. But even assuming that the plans were in hand and the votes available, it isn’t clear that it would be possible to generate many good jobs in places where workers and customers alike feel, and often are, unsafe. And while racial animus against African Americans has multiple sustaining causes, surely the crime-rate differential is important among them; it isn’t only white cab drivers who prefer not to respond to street hails from young black men.

The need to punish past offenses and control future offenses more effectively—and with much more sparing use of jails and prisons—could hardly be clearer. After a quarter century of explosive growth and almost two decades of decline, crime rates are nearly back to their 1965 levels. We should make it a national goal to keep those crime rates moving down, while getting back to the 1965 level of incarceration. That would merely put us back in line both with our own history and with incarceration rates in other advanced democracies, but it’s also an ambitious goal; it would represent an 80 percent decrease from our current incarceration rates.

The good news is that we already know how to do all of this, while spending less money and imposing vastly less suffering than we do today. The key is understanding how punishment works to change behavior among actual offenders, rather than among the rational economic actors imagined by some theorists of deterrence. The conservative goal of making sure that “crime does not pay” was achieved years ago; blue-collar offending—theft and retail drug dealing—offers a remarkably bad bargain to those who engage in it, even if the alternative is working at McDonald’s. Low-level crime, and the drug abuse that often accompanies it, derives more from ill-controlled shortsighted impulse than from rational calculation. If the problem were deterring perfectly rational offenders enticed into crime by inadequate punishments, continuing to ratchet up the severity of punishment while applying it only sporadically might work. But that’s not the problem we actually have: legions of poorly self-controlled offenders making a mess of their own lives and devastating their neighborhoods because they can’t shape their current behavior around their long-term plans, or don’t have long-term plans at all.

Theory and evidence agree: punishment that is swift, certain, but not severe will control the vast bulk of offending behavior. Severity is not only a poor substitute for swiftness and certainty, it is also the enemy of both. The more severe a sanction is, the less frequently it can be administered. (Prison cells are scarce and expensive, and the steeper the punishment, the more time consuming the processes required to avoid gross miscarriages of justice.)

Consider a longtime cocaine, heroin, or methamphetamine user on felony probation for burglary. Under current standard procedure, he will appear once a month for an appointment with a probation officer and perhaps receive a drug test. If he fails to appear, tests “dirty,” or violates some other condition of probation—of which there are many—he is theoretically subject to having his probation revoked and being sent off to prison for the entire length of his probation term. That threat, if it were genuine, might keep him on the straight and narrow. But in fact he knows perfectly well that his probation officer has a caseload of nearly 200, that many of his fellow probationers are also breaking the rules, and that writing up a revocation notice on each violation would take more hours than the overworked PO has in her workweek. Therefore, he can be confident that the response to a violation will be a warning rather than a punishment. And he is therefore likely to break the rules—typically, by using his favorite drug—again and again.

However, if he keeps it up, the PO may eventually decide that six or eight or ten warnings are enough, and finally write that revocation motion. With such a long list of violations, the probationer may well find himself on his way to (or back to) prison. And all he will have learned from the experience is that it wasn’t his lucky day, or that his PO was in a bad mood or had it in for him personally; after all, he’s going “inside” for something he was doing with impunity before, and that some of his fellow probationers are still getting away with. And that’s consistent with his entire life experience; the random nature of the system simply reproduces the sporadically severe discipline he likely received at home and the chaotic conditions in his social environment. His whole life has been spent having very little control over what happens to him, with only a weak correlation between his current behavior and consequences. If the drug-using burglar doesn’t go back to prison for violating probation, he’s likely to do so the next time he gets caught for one of the burglaries that he commits to support his habit, and that he probably won’t stop committing as long as he keeps using.

It would be hard to imagine a system that could combine more punishment with less effective social control. It’s no surprise that so many people wind up what we could call “doing life in prison on the installment plan.” A parent who acted the way the probation system acts—letting most misconduct go unpunished, and occasionally lashing out with ferocious punishments—would be called both neglectful and abusive.

But it turns out to be possible to make “swift-certain-not-severe” sanctions work, by giving each offender a clear and explicit warning of exactly what’s going to happen every time he gets caught breaking a rule. Ideally, that warning would be accompanied by sincere expressions of goodwill and confidence that the offender can muster the capacity to comply.

The best-publicized program built on this set of principles is the HOPE program in Honolulu, which requires random drug tests of probationers and, for those who fail, an immediate short stint (typically two days) in jail, with no exceptions. The SWIFT program in Texas, the WISP program in Seattle, the Swift and Sure program in Michigan, and Sobriety 24/7 in South Dakota all work the same way, and all have the same results: drastic reduction in illicit-drug use (or, in the case of 24/7, alcohol abuse), reoffending, revocation, and time behind bars.

There’s nothing surprising about the fact that this approach works—it’s simply the application of well-known behavioral principles to a fairly straightforward problem. What is surprising is how well it works. In Hawaii, HOPE clients are mostly longtime criminally active drug users with a mean of seventeen prior arrests. A drug treatment program would be delighted if it could get 20 percent of such a population into recovery—and most would quickly drop out and go back to drug use. But in a carefully done randomized controlled trial with 500 subjects, eight out of ten assigned to the HOPE program finished the first year of the program in compliance and drug free for at least three months, with no rearrest. Most of them either never had a missed or dirty test (which would have led to a forty-eight-hour jail stay) or had only one such incident. That suggests that more than mere deterrence is at work; HOPE clients seem to be gaining the ability to control their own behavior.

But it’s hard to organize a system with many moving parts (judges, court staffs, probation officer, their supervisors, police, prosecutors, defense attorneys, and jailers) to make the threats in a convincing but respectful way and then deliver on them, right away and every time. The temptation on the part of probation officers and judges to cut an erring probationer some slack “just this once” can be disastrous; when consistency is the name of the game, mercy is toxic. That’s part of the reason to make the sanctions mild. The most successful current programs mostly use a couple of days in jail, but it’s possible that home confinement or a curfew might work just as well. What’s essential is making the consequence inevitable and immediate.

Attempting to impose such discipline on all probationers at once is a recipe for failure. The violation rate outstrips the capacity of the system to deliver on its threats, and the result is a loss of credibility and an upward spiral of violation rates. The recipe for success is to start with a limited number of offenders, establish the credibility of the threat, and then increase the number of offenders subject to the system slowly enough that the demand for sanctions, in the form of violations, never exceeds the supply.

The strategy of concentrating on a small group first exploits a central, but poorly understood, phenomenon: positive feedback in rates of offending. In a group of generally well-behaved individuals, enforcement can concentrate on a small number of miscreants, delivering swift and certain sanctions, and the resulting high probability that any offense will lead to punishment will make misbehavior an unattractive option. When punishment becomes sufficiently certain and swift, the level of offending goes down, making the high certainty of punishment (conditional on offending) consistent with low levels of actual punishment.

HOPE, SWIFT, and 24/7 all apply to probationers: people put under community supervision, instead of being incarcerated. WISP addresses parolees under supervision after incarceration. Offenders released on bail or their own recognizance while awaiting trial are also a special case, because they are still legally presumed innocent. Juvenile probationers present different challenges: how, for instance, do you punish them without disrupting their schooling and family life? All of these groups are good candidates for smart community supervision.

Unlike resource-intensive programs such as drug courts, programs built around “swift-certain-not-severe” sanctions could be done at a scale large enough to transform the system. It’s logical to think that adding the right services to tighter supervision would further improve the results. That research has yet to be done. But the contrast between the impressive results of such supervision-oriented programs and the much-less-impressive results of services-oriented “reentry” programs ought to generate a rethinking of the services-only approach.

Low-level offending—predatory crimes such as burglary and retail drug dealing—is not only causally linked to substance abuse; they also share a common mechanism. Both involve actions with modest short-term benefits and large, but deferred and sporadic, costs to the people who engage in them. We know much less than we need to know about “self-command”—the capacity to shape current behavior to reflect long-term interests and goals. Early childhood education might usefully focus as much on enhancing self-command as on narrowly “cognitive” goals. If it succeeded in that task, it might help keep those children from later coming into contact with the criminal justice system, to everyone’s benefit. The Nurse-Family Partnership program, which “coaches” at-risk mothers in parenting technique, and the Good Behavior Game, which enlists the competitive team spirit in the service of classroom discipline, both seem to substantially reduce children’s contact with the criminal justice system. So does reducing the environmental factors that damage the self-command mechanisms in the brain, such as lead in housing and the soil and fetal exposure to alcohol.

This is mostly common sense, not rocket science. A sensible crime-control agenda would satisfy neither the conservative impulse to punish as many people as possible as severely as possible nor the liberal impulse to substitute services for coercion and social reform for law enforcement. Liberals will have to swallow the idea that improved coercion is as necessary as improved conditions. Conservatives will have to swallow the ideas that punishment is a cost and not a benefit and that the measure of the efficacy of a threat is how often it does not need to be carried out. Fortunately, the Right on Crime organization has such luminaries as Newt Gingrich, Grover Norquist, William Bennett, and Edwin Meese signed on to those principles. (See David Dagan and Steven M. Teles, “The Conservative War on Prisons,” Washington Monthly, November/December 2012.) It will be more difficult but not impossible to convince conservatives that providing services to actual and potential offenders can in some circumstances control crime more cost-effectively than law enforcement.

Criminal justice institutions need to give crime control priority over institutional comfort and habit. Public and nonprofit agencies that do not have crime control in their mission statements need to acknowledge that they are nonetheless in the crime-control business, whenever their actions and omissions can make the crime problem better or worse.

The bad news is that current policies leave us with unnecessarily and unforgivably high levels of both crime and incarceration. The good news is that we now know how to do better.

Click here to read more from our Jan/Feb 2013 cover package “Race, History, and Obama’s Second Term.”

Mark Kleiman

Mark Kleiman is a professor of public policy at the New York University Marron Institute.