Managing Uncertainty in Experts: What are the Key Issues?

Introduction

Conducting a medico-legal interview and preparing a comprehensive report for the court, consistent with CPR rules and culture, involves a rigorous approach to data collection. Experts are aware of the importance of working with evidential complexity and differential sources of information (claimant self report; medical records; changing course of symptoms over time). As a result of this challenging task, experts in civil proceedings have to develop their own notions of what ‘certainty’ means in theory and practice in this context.

The concept of ‘certainty’

The court need to demonstrate that expert opinion rests upon a properly laid evidential foundation (Drogin at al, 2012) which can withstand cross examination to ‘discover the truth’. The use of a second expert (defendant) and the Joint Opinion process is used to buttress, impugn or confirm the legitimacy of an opinion. Phrases typically used to represent levels of evidential certainty include: ‘on the balance of probability’, or ‘reasonable degree of certainty’.

This paper is the first of two papers which explore the concept of ‘certainty’ in expert deliberations, in order to help e xperts clarify the quality of their opinion with ‘true conviction as opposed to vaguely understood incantations’ (Drogin et al 2015, p 351).

From our experience undertaking e xtensive medico legal assessments in our Personal Injury clinics, the authors shared their views about certainty and uncertainty and the typical areas of their medico-legal work which caused most uncertainty.

They described their views of certainty as involving both cognitive process (e.g. logical, illogical, confusion when faced with inconsistent data) and emotional
processes (e.g. unhappy and anxious) about inability to be clear about the task in hand (e.g. diagnosis). In our view, the top 13 causes of uncertainty were:

1. Inconsistency between sources of data (e.g. self report/GP records)

2. Unusual symptoms reported (inconsistent with widely understood clusters of symptoms)

3. Magnified or exaggerated symptom self-report.

4. Borderline symptoms (clinically significant/ diagnostic or not)

5. Differential disruption (social/psychological but not occupational disruption)

6. Lack of confirmatory GP attendance history

7. Differential diagnosis between clinicians

8. Interpreting unreliable information from claimants

9. Understanding the interaction between pain and mood

10. Proportionality and multiple causality (e.g. two accidents)

11.Effects of litigation-maintaining symptoms

12. Interpreting poor response to therapy

13. Differentiating between different diagnosis

We intend to develop a typology of areas of uncertainty in due course but a preliminary model consistent with the above views is shown below:

• Diagnosis

• Causation

• Prognosis

• Self-report unreliability

• Between-source inconsistency

Developing or changing my opinion? In everyday decision-making, individuals tend to initially gather relevant information and formulate an opinion based on that information. They then are open to modifying this opinion when additional information is available.

In litigation, the same situation prevails although with some caveats.

In his blog article ‘The Inconsistency of experts and the failure to apply the Legal Tests’, Giles Eyre, barrister, quotes a recent High Court Case (Pullen v Basildon and Thurrock University Hospitals (2015)) EWHC 3134 where an expert who changes his or her opinion is at risk of being found an unreliable expert. He states that a change in opinion needs rational justification, whether it occurs after own side formal questions, conference with counsel or CPR 35-6 questions from the other side. The judge in this case, where one expert altered his view, noted that the court can accommodate a frank change of opinion as long as this is rational, credible and logical and occurs at the earliest opportunity. In this circumstance, a changed opinion can carry weight in court (Eyre, 2015).

Here we see the tension between the initial search for certainty and subsequent need to manage uncertainty.

Researching Intolerance of Uncertainty

In addition to eliciting qualitative views from experts about their dilemmas in medico-legal work, we plan to assess quantitatively how experts tolerate uncertainty. To begin with, we want to explore whether experts vary in terms of their personality and general intolerance of uncertainty, before then addressing the same variable specifically in their medico-legal practice.

A short form scale based on the Intolerance of Uncertainty scale (Freeston et al, 1994) was developed by Carleton et al (2007) which measures responses to
uncertainty, ambiguous situations and the future. The 12 items of the short form are each rated on a 5-point Likert Scale ranging from 1 (not at all characteristic of me) to 5 (entirely characteristic of me). For example, statements included:

1. Unforeseen events upset me greatly

2. It frustrates me not having all the information I need

We have adapted this scale (Koch and Newns, 2015) to specifically tease out views on uncertainty when undertaking medico-legal work. For example, statements included: -

1. Uncertainty keeps me from having confidence in my opinion giving

2. I dislike being take by surprise e.g. newinformation, part 35 questions

We anticipate sampling a group of 25 – 30 experts who regularly undertake medico-legal assessments and aim to report on results in approximately 6 month’s time.

Weighing up Evidence and coping with uncertainty There is a broad consensus amongst lawyers and experts that experts should use the scientific method to formulate and test alternative hypotheses, using evidence from multiple sources to seek collaborative or disconfirming evidence (Milchman, 2015). Scientific or logical reasoning can identify limitations in evidence, using consistency as the basis for reasoning from evidence to opinion.

The problem with aiming for logical answers is that inconsistency will occur! How should experts deal with case-specific evidence that is inconsistent?

Important principles (Milchman, 2015) are:

1. The meaning of inconsistent evidence, no matter how little there is, may diminish the weight of the consistent evidence, no matter how much there is.

2. An expert opinion has the highest level of validity:

a. When it accounts for consistent and inconsistent evidence.

b. When most of the evidence supports the opinion, no evidence contradicts it, and inconsistent evidence can be explained so that it no longer reasonably calls it into question.

This ‘Step-Down’ analysis attempts to assess internal validity underpinning an expert’s opinion and considers the relationship between consistent, inconsistent, and contradictory case-specific evidence.

Crucially, it counteracts claims that expert opinions are based on intuition, are inscrutable, and not subject to rational analysis. As such it makes expert reasoning more transparent (Milchman, 2015).

Conclusion
This article sets the scene for qualitative and quantitative research we plan for 2016. Experts have the skills to manage and tolerate the uncertainty and either use it, like a therapist, to identify problems or inconsistencies in what they are being told or as a sign that additional information is required. It is anticipated that this research will also have training and CPD implications.

References

Carleton R.N, Norton M.A, and Asmundson G.J.G (2007) Fearing the unknown: A short form of the intolerance of uncertainty scale. J. Amx. Disorders 21, 105 – 117

Drogen E.Y, Commons M.L, Gutheil T.G, Meyer D.J and Norris D.M (2012) ‘Certainty’ and expert mental health opinions in legal proceedings. International Journal Law and Psychiatry, 35, 348 – 353

Eyre G (2015) The Inconsistency of experts and the failure to apply the legal tests. Medico-legal Minder. 11.11.2015

Freeston M.H., Rheaume J, Letarte H, Dugas M.J. and Ladouceur R (1994), Why do people worry? Personality and Individual differences. 17, 79-802

Koch HCH and Newns K (2015) Development of Intolerance of Uncertainty in Medical-legal opinion forming. Unpublished test. Cheltenham

Milchman M.D (2015) Weighing evidence in psychological experts opinion. Paper presented to IALMH conference. Vienna