Forensic Lip Reading

Lip reading, or speech reading, is the skill of using one sense that was meant for another. In place of heard speech, lip shapes are seen, decoded by one’s brain and knowledge of language, and translated so that they make sense. A lip reader will watch a person’s lips, facial expressions, eyes, gestures, and body language, and if possible they will use context to clue themselves into the topic. Context is very important in lip reading but this is often missing in forensic lip reading.

Forensic lip reading is the practice of applying lip reading skills to (typically silent) video footage where no context is given since an airlock must be present,i.e. no information about a case should be given to the lip reader in order to avoid any contamination inpresenting the material.

Forensic lip reading is an art, not a science. It is a skill that is difficult to do well - it’s much more difficult to lip read a small flat face on a TV screen than a real life 3D person. In forensic lip reading, the lip reader will replay a video for a few words at a time, typically replaying 2 or 3 seconds, for several cycles - playing those seconds up to 50 times is common. Less skilled lip readers will look at a video perhaps 3 or 4 times,and give up too quickly. If a sentence is not recognisable immediately, it does take quite some time to unpick words and work out what is seen and grasp the context. It can take on average one hour to work through one minute of video.

Why does lip reading ability vary so much from person to person?

Speech is designed primarily to be heard, not viewed.Many of the critical aspects of speech are hidden from view. Most consonants are produced by actions of the tongue inside the oral cavity (g,d,t,y,r,s,n,k,sh,s,j,z)and not by visible actions of tongue, lips, teeth (m/p,f/v, th). Lip shapes do not always reflect the speech sounds being made, but can anticipate or follow them: for example, the mouth shape for the final ‘th’varies for saying ‘tooth’ and ‘teeth’ because of the preceding vowel - watch yourself saying these in the mirror.

For these reasons, it is often concluded that onlyaround 30% of spoken English is lip readable. However,this is misleading as it is too general. Firstly, people vary in their lip reading ability, and better lip readers can see more distinctive speech elements than poor lip readers. Secondly, some distinctions between speech actions are highly visible to good and poor lip readers alike. For example, when presented with a full range of seen syllables, up to 60% accuracy can be achieved by naïve hearing perceivers, as longas the dynamic characteristics of the speaking face are well captured (i.e. the speaker’s face shows natural movement) and the critical parts of the mouth can be seen.

Someone who is born deaf and relies on lip reading to communicate, and has built life-long experience of reading lips, will be a much better lip reader than someone who became deafened in later life, or primarily uses sign language, or is hard of hearing and can hear a bit, or is hearing (perhaps with deaf relatives).If you are looking for a lip reader who is truly an expert, look for someone who has high cognitive skills and lifelong experience of reading lips as their sole means of communication, and who is able to holda conversation with you without other communication support being required.

What makes a person a proficient lip reader?

Some lip readers say they can lip read anything, they can lip read upside down and in any language, which is very amusing as this is just not possible. Most people can read lips to some extent but extensive daily practice and a good knowledge of language will improve lip reading skills.

When mentally sorting out the words to grasp the context of what is seen, the lip reader needs to understand which visemes and which words look alike. They can then sort out the context and make sense of whatt hey can see. This is why working with an interpreter and a non-native language is very high risk and not recommended.

Intelligence, alertness, and a good working and short term memory are essential so that shapes can be recalled seconds and minutes later, to help unscramble what is seen and make it into a coherent whole. When tiredness overcomes the lip reader, this often means they cannot lip read (and therefore communicate), as working on forensic lip reading is so labour intensive.

A more experienced lip reader will be more skilled at distinguishing between very similar word shapes,homophenes, and in picking up words which are difficultto see. They will also be better at making use of the available context to work out the possibilities of what a word could be. Sometimes a mental block can occur or local language knowledge can get in the way when working, and for this reason it is best practice to have two forensic lip readers work together. This also reduces the likelihood of a less skilled lip reader making guesses at what they think they see

.How do you know if a lip reader is good at reading lips?

Have a conversation with them over Skype video and switch off the sound, or meet them face to face and don’t use your voice. If they can’t lip read you and they require communication support to understand you (e.g. sign language interpreter, lipspeaker, speech to text reporter) and they ask you to speak slowly, or if they are able to hear you and are listening, they aren’t highly skilled lip readers. A lip reader will require communication support in the courtroom as the judge is more than five feet away and they may mumble or not look at them when speaking.There is no official qualification available for lip readers,therefore assessment of skill is very much up to observation and individual judgement.

Which factors make lip reading a video more difficult?

Video material submitted to us is not always suitable for lip reading. Many factors affect the lip readability of a video so the outcome is rarely a verbatim translation,except where the video quality and filming isexceptionally good and the lip reader is exceptionally skilled. There are a number of factors which affect forensic lip reading work

.Video quality

Broadcast quality and a stable recording is much easier to work with than footage which is blurry, shaky and grainy. We get a lot of videos from surveillance agencies and the quality varies widely. A CCTV video came in which was the best quality we had ever seen,however the people speaking were too far away from the camera – which meant we couldn’t see their faces.Video enhancement may help in some cases where the footage is slightly blurry. If we are able to zoom in and slow down playback, this can be very helpful.Lighting and shadows also affect lip reading.

Camera angles and distance

Ideally the speaker should be facing the camera full on.We have been sent CCTV footage with the cameras pointing downwards from a great height and al lwe could see were the tops of peoples’ heads. The speakers being lip read should be as close to the camera as possible. It is very difficult to read lips measuring 3 millimetres and zooming in can pixelate the video. Trying to lip read people when they are walking or running, or having a camera filming sideways,will add an extra layer of difficulty for the lip reader.

The speakers

Mumbling, a lisp and stuttering make lipreading more difficult. If the speaker does not have an expressive face or wears sunglasses, there is no emotionand this makes lip reading much more difficult - a good example of an expressionless face is Sven-Göran Eriksson. Facial hair, smoking, eating and drinking can get in the way of a clear view.

In real life, people move, they walk around, they look away, someone walks between the camera and speaker– so a lip reader’s transcript will typically have several gaps where words cannot be seen or worked out.

People have accents and talk fast. This is a challenge even for a skilled lip reader, necessitating more rewinding to catch those milliseconds where a whole word had been said. The lip reader has to work out the gist of the conversation from all the bits they are analysing, build up the context, then fill in any missing words that match what they can see, in fast running speech.

One memorable job was a CCTV clip of two people having a conversation, the camera was only a fewi nches away and was very clear – unfortunately both speakers had their backs to the camera and we could only see the backs of their heads.

Is lipreading evidence admissible in the courts?

In R v Luttrell (2004) the Court of Appeal set out the test for admissibility of expert evidence on new are as such as lipreading. The following criteria are includedi n the test;

i) The evidence has to be relevant and outside of the jury's experience

ii) The expert has to be qualified to express an opinion

iii) The judge must give appropriate warnings about the evidence

The Court of Appeal held that it was not necessary that any results should be verifiable under cross examination.

Research to date has focussed on watching a single speaker’s speech actions. The skills of lip readers have been scrutinised primarily to help improve communication between the deaf lip reader and the speaking hearing population. Tests have been developed to assess individual differences in lip reading skill, the most common one being the Bamford-Kowal-Bench(BKB) test however this is only a very basic test. Forensic lip reading is a hefty step up from daily lip reading inability and confidence. It should be noted however that: 

F There is no reliable scientific research data related to lip reading conversations between different speakers.

F There are no published studies of expert forensic lip readers’ skills in relation to information gathering requirements (transcript preparation,accuracy and confidence).

F There are no published studies relating individual performance on a specific test of lip reading with performance in relation to information gathering requirements.

As inferences have to be drawn from the published and reported studies which arose in a different context,caution and an empirical approach should be applied in considering how lip reading may be used for information gathering purposes.