Evaluating Restorative Justice Programmes

Introduction

My intention in this paper is to add a cautionary note to the way in which the ‘success’ of restorative justice is currently measured. My basic argument is this:

1. There are certain restorative processes which, for various reasons, we don’t (or don’t know how to) measure that are essential to a restorative outcome. 
2. Evaluation reports of restorative justice programs which do not acknowledge this limitation are therefore likely to have a distorting effect on the way such programs are designed and operated: specifically, those restorative processes which are not (or cannot yet be) measured are likely to be neglected or downplayed, leading to an outcome which is either not fully restorative or counter-restorative.

I will begin the paper by trying to encapsulate, in very general terms, the three processes which are essential to restorative justice, what must happen for restorative justice to occur. I will then look at how victims and offenders are affected when a restorative justice program neglects or downplays just one of these processes, and then go on to suggest that this phenomenon may have a great deal to do with the way in which programs are currently evaluated. 

1. What is essential to restorative justice?

The primary site of restorative justice is not an adversarial court of law, a prison cell, a boot-camp, or an execution chamber. It involves a mediated encounter between those directly involved in or affected by the crime: the victim, the offender, family members, and community representatives.

The principal aim of these encounters is to facilitate the following three processes.

• Reconciliation: where the victim and offender—in the social rituals of apology and forgiveness—(i) offer and receive the value and respect owed in virtue of their intrinsic human dignity and worth, and (ii) engage in a mutual condemnation of the criminal act, whilst ceremonially ‘casting off’ or decertifying the offender’s deviant or blameworthy moral status.
• Reparation: where the offender takes due responsibility for the crime by ‘making good’ the material harm done to the victim: that is, by agreeing to provide a fair and mutually acceptable form of restitution and/or compensation.

And, as an ongoing consequence of reconciliation and reparation:

• Transformation: where the individuals and communities concerned experience some degree of liberation from the conditions that perpetuate the cycle of violence, aggression and domination exemplified in criminal behaviour: for example, by overcoming the negative emotions of humiliation, fear and hatred, and by advancing the alleviation of degradation, oppression and stigmatization which characterize existing socio-political structures and relations.

2. What happens when there is reparation without remorse?

Let me say first of all that I do not want to deny the essential role of reparation in restorative justice: victims are entitled to have “the value of property stolen or destroyed returned to them”.[1]  Nor do I want to deny that the act of restitution can, in some cases, serve as a symbolic gesture of reconciliation. However, I do want to suggest that programs which emphasize restitution settlements—to the neglect of reconciliation—can give rise to several counter-restorative outcomes.[2]

The first example of this is where the lack of remorse in the offender ends up re-victimizing the victim, even when reparation has taken place. Mark Umbreit, for example, has found that, where the offender remained unrepentant, victims tended to view their restitution agreement with resentment, dissatisfaction, and a sense of arbitrariness (e.g. ‘I felt he wasn’t owning up to it.’; ‘He just slouched all the way down and just sat and half‑heartedly gave answers’.”[3]). As Marshall has put it:

“It is not possible to carry out fruitful mediation without dealing with underlying feelings. A material agreement without this will be super­ficial and of little meaning to the parties. Mediators should be prepared to gain the skills necessary for ventilation and expression of grievances, not merely for their direct therapeutic benefits, but also because the ultimate settlement will have more content and value.” [4]

A second example is where an offender does not think of the reparation agreement as an expression of their genuine remorse, or of their desire to ‘make things right’. As far as they are concerned, it is all about the victim having a say in what kind of punishment they receive.[5] The result is just what we have come to expect from retributive institutions: offenders fake their way through the program, all the while reinforcing their sub-cultural identity as a ‘victim’ of the system. Blagg’s 1985 study, for instance, found that offenders who were merely expected to make restitution to the victim and were given no encouragement or opportunity to express genuine remorse, reported their perception of the encounter in this way:

 “they were punished by an authority figure; they were powerless to prevent the process; they acquiesced; they then, in order to retain peer-group status and keep their egos intact, retrospectively recreated the encounter as one in which sullen obeisance was transformed into heroic resistance”[6]

3.  Why have researchers focused on reparation agreements?

One explanation for the concentration on reparation, might be found in the methodologies typically used to evaluate restorative programs. The majority of published (and unpublished) evaluative research has focused almost exclusively on the social service features of victim-offender encounters.[7] In other words, the evaluative criteria of this research has typically been restricted to delivery efficiency (e.g. costs per case), effort (e.g. caseloads per mediator), and outcome (e.g. percentage of agreements, satisfaction of dispu­tants, restitution compliance rates).[8]

There are several reasons for this restriction. First, to justify their existence and funding, restorative justice programs have had to appeal to the persuasive power of utilitarian or economic rationalism:[9] victim-offender encounters are advanced as preferable alternatives to the traditional criminal justice process on the grounds that (i) they will decrease court caseloads, the prisoner population, and recidivism rates; and (ii) they will increase the percentage of restitution settlements and victim/offender satisfaction.

Second, most of the data relevant to service‑delivery criteria is comparatively easy to collect: minimal requirements for program management will involve keeping records of costs per case, caseloads, referral sources, types of cases, percentage of settlements reached, and, with a little more effort, percentage of restitution compliance and participant satisfaction. Third, it is, as a consequence, relatively cheaper to produce program evaluations using service-delivery data. Finally, the audience for which these evaluations are primarily designed—funding agencies, policy makers, and criminal justice professionals—do not generally require, and would not necessarily appreciate or acknowledge more qualitative or substantive data.[10]  

4.  What does service-delivery data tell us about restorative processes?

The problem is that, even if the service-delivery criteria were shown to be satisfied, such an evaluation would tell us almost nothing about the more substantive claims made for victim-offender encounters:[11] how would we know, on the basis of service-delivery data, whether a particular encounter has, indeed, ‘given participants access to a higher quality of justice’, ‘evoked genuine remorse in the offender’, ‘enabled the victim to overcome her resentment, fear and negative self-identity’, ‘repaired the social bonds’, ‘shamed the offender within a continuum of love and respect’, ‘decertified his deviant status’, and so on? But until such information is forthcoming—that is, in non-anecdotal form—there remains little basis for the claim that victim-offender encounters are theoretically grounded in the social and experiential reality of its participants. As Umbreit has put it:

“The ultimate strength of any social theory is to be found in how accurately it captures the reality of people who are subject to it. Restorative justice theory makes bold claims about the needs of people affected by crime within community structures. Its validity as a new social theory must be grounded in empirical evidence offered by those most affected by crime—victims and offenders.”[12]

To illustrate this problem, take the criterion of restitution agreement percentages. This is perhaps the most widely used source of evidence for the success of victim-offender encounters. But what does the fact of an agreement tell us about the more substantive issues?

• First, if the parties have agreed to participate in a mediation session, they will already be sufficiently motivated to achieve some kind of settlement. Second, an agreement may vary to an enormous degree in terms of its significance for the participants; and this may be impossible to determine by reference either to the fact of an agreement or to its content. For example, the ‘settlement’ may involve a simple apology, substantial monetary restitution, a signed pledge to perform community service, an agreement ‘to have nothing to do with each another from this point on’, and so forth.
• The problem is that any one of these forms of reparation may represent a substantial breakthrough in terms of reconciliation. On the other hand, the agreements may be token offerings to ‘get the thing over with’, lacking any reconciliatory purpose. In sum, an encounter might be classified as ‘a success’ on service-delivery grounds, and yet fail entirely to accomplish what should be one of the primary goals of Restorative Justice, reconciliation. Alternatively, it may be classed as a ‘failure’ due to the lack of any significant reparation settlement, and yet the participants may have nevertheless experienced reconciliation.[13]

5.  What is the effect of focusing on reparation as a criterion of ‘success’?

There is growing evidence that restorative justice is a powerful alternative to the traditional criminal justice system: where everything else is failing, restorative justice programs somehow seem to be ‘working’. For those of us who have observed mediation, conferencing or circles first-hand, we know that the claims of restorative justice ring true: for the most part, these encounters really do give participants access to ‘a higher quality of justice’, they do somehow manage to ‘shame the offender within a continuum of love and respect’, they do ‘enable victims to experience forgiveness’.

The problem is that we do not yet know how to test these sorts of claims.  We don’t really know what kind of data would show that ‘a victim experienced a sense of forgiveness during an encounter with his or her offender’. But our suspicion is that— whatever data is relevant—it would be far too difficult and expensive to collect and analyse on a large scale; and the results would, in any case, be far too complex for most stake-holders to digest. Fortunately, there are certain things we can measure which do not have these sorts of obstacles, such as recidivism rates, victim satisfaction and restitution payments. And if we can persuade those who matter, that restorative justice programs are highly successful using such criteria  . . . why not leave it at that?

Well, there is a very good reason why we ought to reject this approach. As I hope to have shown, what we are leaving out in our research—what we don’t (or don’t know how to) measure—is essential to a truly restorative outcome. And we should acknowledge this limitation. Otherwise, stakeholders may look at the research and walk away with a profoundly distorted understanding of what counts as a successful restorative justice program. For instance, they may think that restorative justice is primarily a means of addressing minor property crimes, simply because our research has focused on the fact that restorative programs are more likely than the court system to achieve restitution settlements. Again, they might, for similar reasons, think that restorative justice can be achieved merely by (a) including victim statements in court decisions, or (b) getting prisoners to contribute to a victim-reparation fund, or (c) setting up a ‘truth commission’, where the offender is only required to give a full disclosure of their crime in return for amnesty, or even (d) holding a victim-offender encounter where the sole purpose is to extract a restitution agreement from the offender. These may indeed serve as part of a restorative process; but they are not explicitly designed to facilitate reconciliation. If reconciliation happens to occur, as it sometimes does, it is quite inadvertent and unexpected. In other words, because of what they leave out by design, these programs, on their own, are not likely to result in a truly restorative outcome. As we have seen, some may even be counter-restorative. 

6.  Conclusion

Restorative justice is rich, complex and multi-dimensional: it must, at some point, involve reconciliation as much as it involves both reparation and the slow but steady transformation of individuals and communities. I have tried to show that how we measure these restorative processes does make a difference. As Gordon Bazemore has put it, ‘You get what you measure’.  And this is my fear.  But I am also optimistic. My hope is that, someday, we will be able to measure what we get.

BIBLIOGRAPHY

Blagg, H. (1985) “Reparation and Justice for juveniles” British Journal of Criminology, 25, 267-279.

Davis, G., Messmer, H. & Umbreit, M. (1992) Making Amends: Mediation and Reparation in Criminal Justice. London and New York: Routledge.

Gehm, J. (1990) “Mediated victim‑offender restitution agreements: An exploratory analysis of factors related to victim participation”, in B. Galaway & J. Hudson (eds.) Criminal Justice, Restitution and Reconciliation. Monsey NY: Criminal Justice Press, pp. 177‑182.

Lowry, K (1993) “Evaluation of Community­ Justice Programs”, in S. E. Merry & N. Milner (eds.) The Possibility of Popular Justice: A Case Study of Community Mediation in the United States. Ann Arbor, MI: The University of Michigan Press, pp. 89-122.

Marshall, T. & Merry, S. (1990) Crime and Accountability—Victim/Offender Mediation in Practice. Home Office, HMSO, London.

Retzinger, S. & Scheff, T. (1996) Strategy for community conferences: Emotions and social bonds. In B. Galaway & J. Hudson (eds.) Restorative Justice: International Perspectives Monesy, NY: Criminal Justice Press.

Umbreit, M. (1990) “The meaning of fairness to burglary victims”, in B. Galaway & J. Hudson (eds.) Criminal Justice, Restitution, and Reconciliation. Monsey, NY: Criminal Justice Press.

Umbreit, M. (1994). Victim Meets Offender: The Impact of Restorative Justice and Mediation. Monsey, New York: Willow Tree Press, Inc.

Van Ness, D. & Strong, K. H. (1997) Restoring Justice. Cincinnati: Anderson Publishing Co.

Wright, M. (1991) Justice for Victims and Offenders: A Restorative Response to Crime. Bristol, PA: Open University Press. 

[1] “[E]xperimental schemes may have gone too far in stressing the emotional and being overly ready to dismiss the material side of reparation. This shift of emphasis was needed to combat the general bias of criminal justice towards the material and its neglect of victims’ real needs, but it should not be taken so far that it effectively denies a victim’s basic right to have the value of property stolen or destroyed returned to them. If this right is to be waived, it must be at the discretion of the victim and no one else.” (Marshall 1990, 100); “[V]ictims might be interested in material reparation, but because mediators were preoccupied with resolution through talking (and symbolic gestures of reconciliation), victims’ preference for getting their money back might remain unvoiced and unrecognized.” (Davis et.al.1992, 457).

[2] This is not a rare phenomenon: “Programs’ personnel tend to characterize success in relation to rates of contract completion” (Gehm 1990, 179); See also Van Ness & Strong 1997, 71; Retzinger & Scheff 1996, 317.

[3] Umbreit 1990, 56

[4]  Marshall 1990, 98.

[5]  “[R]eparation can be a highly complex process requiring skill and sensitive handling and . . . its value as a lesson for juveniles may well be lost if it merely replicates the punishment paradigm, albeit by a more insidious route” (Blagg 1985); “discussion of material things . . . tended to appear more like victims determining the offender’s degree of punishment, rather than the determination of what was merely the victim’s due.” (Marshall 1990, 99).

[6] Blagg 1985, quoted in Davis et.al.1992, 143.

[7] A good example of the service-delivery evaluative framework is Mark Umbreit’s “two-and-one-half-year study of victim-offender mediation programs in California, Minnesota, New Mexico and Texas.” (p. xi) Without wishing to deny its value, the study’s research questions were evidently designed to evaluate the satisfaction of one of more of the three types of social service criteria (Umbreit 1994, 31-32). I select Umbreit largely because his research methodology is both well-known and representative of the majority of research on victim-offender mediation.

[8] Lowry 1993, 117.

[9] “For the sake of maintaining the confidence of [funding or government] agencies or of the general public, practitioners (even if there are no doubts in their own minds) will . . . need to supply some evidence that worthwhile progress towards ultimate goals is being made. . . . Questions of economy and cost-effectiveness, or efficiency, are . . . prominent at this stage.” (Marshall & Merry 1990, 16-17).

[10]   Lowry 1993, 119. “Sentencers . . . may be more easily persuaded to take account of material outcomes—compensation paid, reparative work carried out—than the metaphysic of empathy and forgiveness. . . and may lead a scheme to place undue emphasis on the material agreement.” (Marshall & Merry 1990, 31); “where [mediation] schemes are dependent on the goodwill of other agencies—as those court-based reparation schemes that seek to influence sentencing decisions may be, when material commitments by the offender to make reparation seem more persuasive to judges than mere expressions of regret, even if the victim may not really desire the first and may find more genuine meaning in the second, or in the encounter itself.” (Marshall & Merry 1990, 25).

[11]   “It is easy to add up the amount of compensation paid or the number of hours of community service worked, but these figures, although they may be useful in impressing the providers of funding, do not necessarily mean very much.” (Wright 1991, 537).

[12]   Umbreit 1994, 6.

[13] “The essence of mediation, according to most who engage in it, is the achievement of understanding, sympathy, catharsis, and the exchange of atonement, on the one side, and forgiveness, on the other. . . . as against a more or less commercial transaction (which may be no more significant than a fine or compensation order imposed by the court).” (Marshall & Merry 1990, 30).