Assessor Skills: The importance of an Experienced, Skilled Assessor
The Challenges Faced in the Assessment of PDOC
Giacino and Kalmar (1997) stated that Vegetative State (VS) is “one of the best known but least understood and most controversial neurological syndromes in medicine”. Several studies have revealed levels of misdiagnosis of the VS patient as high as 43% (Andrews et al 1996, Tresch et al 1991). It is evident from the literature, that detailed, standardised assessments by skilled assessors are required to optimise the accuracy of diagnosis, in order to provide the individual with the best possible opportunity to respond (Wade 1996).
Causes of Misdiagnosis
Given the rarity of this specialist group, there are very few assessors who have seen a sufficient number of these patients to gain a suitable level of experience and expertise (Davis 1991). Therefore, many patients’ responses are overlooked or misinterpreted by assessors who lack experience and will interpret clinical features such as reflex responses as being meaningful, and may also miss potentially significant responses if they possess suboptimal observation skills (Gill-Thwaites 2006).
Childs, Mercer and Childs (1993) and Godbolt et al (2012) consider that misdiagnosis is often due to the lack of extended observation of cognitive awareness by assessors who do not have the time to commit to providing a schedule of frequent assessments. They identified that extended assessment will help to more accurately discriminate between the VS and MCS patient and ensure that a correct and incontrovertible diagnosis of the patient can be reached.
Another factor contributing to misdiagnosis is the failure to systematically involve the family and the carers as active participants and observers of the patient’s behaviours. Wade (1996) acknowledged the value of the contributions made by the family and the carers, and defined the need to interview and formally document their observations of the patient’s behaviours over time. Freeman (1996) stated that the assessor is designated as the judge, who is not solely dependent on his or her own knowledge, but makes use of all the experience available thus allowing a correct verdict to be reached.
In the literature that reports on the rates of misdiagnosis, the assessors identifying the errors in the diagnosis have been experienced (Andrews et al 1996, Tresch et al 1991). In addition, each of the assessment tools recommended for use with the PDOC individual (Seel et al 2010, Royal College of Physicians 2013) has been based on the inter-rater reliability of trained assessors (Giacino et al 2004, Shiel et al 2000, Gill-Thwaites and Munday 1999). This illustrates the importance of not only selecting the correct standardised assessment tool, but also of ensuring that the assessor is experienced and adequately trained to implement the assessment, and to accurately interpret the behaviours exhibited by individuals with PDOC.
Providing Accurate Assessment
All of our consultants are in the elite group of SMART Masters, Expert SMART Assessors and Advanced SMART Assessors, demonstrating that they are highly trained, have extensive clinical experience (in some cases they have assessed in the region of 300 patients each), possess finely-tuned observation skills, teach the SMART course to other SMART assessors, and fully understand the principles of accurate assessment of the PDOC patient.
Gill-Thwaites & Elliott Consultants Ltd is also committed to providing specialised training programmes, which will help to ensure that professionals, carers and family members have the awareness and knowledge to recognise responses, and contribute positively to a thorough assessment of the patient.
Levels of SMART Accreditation
As clients requiring a SMART assessment, it is important to recognise that there are four different levels of SMART accreditation.
Level 4: SMART Master
Accreditation at this level is dedicated solely to those who have invented and/or contributed to the early development of SMART. These SMART Masters will have conducted in excess of 300 SMART assessments. They will have developed the philosophy of SMART by taking a lead role in research pertaining to the validation of SMART and/or designing the SMART courses. SMART Masters conduct SMART assessments for complex medico-legal cases and can act as Expert Witnesses.
Level 3: Expert SMART Assessor
Accreditation at this level supports the practitioner to act as an Expert Witness and to conduct SMART assessments for medico-legal purposes, such as decisions regarding withdrawal of artificial nutrition and hydration. These assessors will have experience involving 30 or more SMART assessments. They will have demonstrated an expert knowledge base of SMART through leading a SMART course, and taking a prominent role in research and/or evidence of publications.
Level 2: Advanced SMART Assessor
Accreditation at this level supports the practitioner to assess patients for medico-legal case management purposes and to establish the individual’s management requirements for the future. These assessors will have experience in carrying out ten or more SMART assessments. They will have demonstrated a good knowledge base of SMART through teaching, research and/or evidence of publications.
Level 1: SMART Assessor
Accreditation at this level supports the practitioner to assess patients in their own clinical practice. These assessors will have conducted between one and ten SMART assessments. At this stage, the assessors will have developed a level of competency to help them to identify awareness, functional potential and treatment programme requirements.
Accredited SMART Assessors Standards of Practice
SMART Assessors at all of these levels are expected to meet the ‘Accredited SMART Assessors Standards of Practice’ (click here to view) in all aspects of their work in order to maintain a consistent, professional approach.
Assessment Tools We Use
The Royal College of Physicians’ guidelines recommend a select range of tools for use with PDOC, in order to help reduce diagnostic error and encourage some degree of consistency for recording longitudinal outcomes. SMART is our tool of preference, however, we can undertake assessments with the Wessex Head Injury Matrix (WHIM) or the JFK Coma Recovery Scale - Revised (CRS-R) as considered appropriate for the individual with a PDOC and your requirements.
The Sensory Modality Assessment and Rehabilitation Tool (SMART)
SMART (Gill-Thwaites and Munday 2004) is an assessment and treatment tool developed to detect awareness, and functional and communicative capacity in the VS and MCS patient, where there have been no consistent or reliable responses elicited and where the individual’s potential function has yet to be fully explored.
Research has shown that 43% of patients who were felt to be in VS were misdiagnosed and were found to be aware following assessment with SMART (Andrews et al 1996).
SMART provides an extended graded assessment of the individual’s level of sensory, motor and communicative responses to a structured and regulated sensory stimulation programme. SMART comprises a ten-session behavioural observation assessment followed by a multi-sensory assessment programme. The SMART assessment is conducted within a one to three-week period, followed by a SMART treatment phase, where indicated. The SMART practitioners attend a 5-day SMART accreditation course to establish their familiarity with the complexity of the behavioural repertoire exhibited in VS and MCS, and to ensure a consistent, competent approach to the assessment protocol.
The family, team and carers are fully involved and their observations are recorded, explored and monitored.
The five-point SMART hierarchical scale defines in which of the five sensory modalities (visual, auditory, tactile, olfactory and gustatory) the patient is able to demonstrate awareness. While there is a range of useful tools available for this patient group, SMART is the only tool designed to address all of the following key areas:
- To identify awareness in the VS patient
- To optimise the individual’s arousal levels to elicit optimal responses
- To identify the widest possible range of communication and functional abilities in the MCS patient, thus ensuring that the identification of the individual’s behaviour and functional and communicative repertoire is fully explored and detection is not left to chance
- To optimise the individual’s progress in the future through clear and logical treatment planning derived from an intensive assessment protocol.
When is SMART indicated?
When the individual with a PDOC:
- Has been identified as being in a VS or MCS following severe brain damage, and requires full assessment to provide objective evidence to contribute to an accurate clinical diagnosis.
- Has not had a confirmed diagnosis, and further assessment is required to identify awareness and the extent of their functional and communicative ability.
- Is in a MCS and has an inconsistent method of communication, or where the pathways for potential communication have not been fully explored.
SMART provides a highly sophisticated assessment, and probably the most definitive evaluation of the level of behavioural response for distinguishing patients in VS from those in MCS, which is the primary purpose for which it was designed. It can also offer useful information on the individual’s strengths and weaknesses in terms of responsiveness, to inform and optimise rehabilitation. SMART possesses a wider spectrum of sensory assessment techniques compared to other existing tools to ensure all responses are fully investigated. As it is a multi-modal assessment, SMART is especially useful for evaluating patients with sensory limitations, such as those with cortical blindness.
What can our assessment protocol offer you and the individual with a PDOC?
- Identification of the type, frequency and meaning of behaviours observed at rest.
- A comprehensive assessment of areas of strength across all of the sensory modalities.
- Comparison of the family and the carers’ observations with the results achieved in SMART.
- A suggested diagnosis based on analysis of the assessment.
- A graded and structured treatment programme incorporating all findings from the assessment and the family/team feedback.
- Identification of the resources, treatment and sensory programmes required to optimise the potential and quality of life of the individual with a PDOC.
Judgment ruling in the Royal Courts of Justice, 28th September 2011
The Hon Mr Justice Baker stipulated that “no such application for an order authorising the withdrawal of ANH (artificial nutrition and hydration) from a patient in VS or MCS should be made unless a SMART assessment (or similarly validated equivalent) has been carried out to provide a diagnosis of the patient’s Disorder of Consciousness”.
W v M and others  EWHC 2443 (Fam), para. 259
The Wessex Head Injury Matrix (WHIM)
The WHIM (Shiel et al 2000) is a systematic observation system, which itemises defined behavioural observations to environmental stimuli. The 62 items are arranged hierarchically to categorise behaviours that may occur either spontaneously or in response to stimulation. Designed to be applied by different members of the multi-disciplinary team, it was developed to monitor changes from coma through to emergence from post-traumatic amnesia in patients with traumatic brain injury.
The JFK Coma Recovery Scale – Revised (CRS-R)
The CRS-R (Giacino, Kalmar and Whyte 2004) has 25 hierarchically arranged items with six subscales (auditory, visual, motor, oromotor, communication and arousal). Scoring is based on the presence or absence of specific behavioural responses to stimuli presented in a standardised manner, from reflexive responses to cognitively facilitated responses. The revised scale was developed to differentiate between the diagnosis of VS and that of MCS. It is widely used in the USA, and is especially useful for tracking patients in the earlier stages of recovery.
Andrews, K., Murphy, L., Munday, R. and Littlewood, C. (1996) Misdiagnosis of the vegetative state: retrospective study in a rehabilitation unit.
British Medical Journal, 313: 13-16.
Childs, N.L., Mercer, W.N. and Childs, H.W. (1993) Accuracy of diagnosis of persistent vegetative state.
Neurology, 43: 1465-1467.
Davis, A.L. (1991) The visual response evaluation: a pilot study of an evaluation tool for assessing visual responses in low-level brain injured patients.
Brain Injury, 5: 315-320.
Freeman, E.A. (1996) New methodology. The coma exit chart: assessing the patient in prolonged coma and in vegetative state.
Brain Injury, 10: 615-624.
Giacino, J.T. and Kalmar, K. (1997) The vegetative and minimally conscious states: a comparison of clinical features and functional outcome.
Journal of Head Trauma Rehabilitation, 12(4): 36-51.
Giacino, J.T., Kalmar, K. and Whyte, J. (2004) The JFK Coma Recovery Scale – Revised: measurement characteristics and diagnostic utility.
Archives of Physical Medicine and Rehabilitation, 85(12): 2020-2029.
Gill-Thwaites, H. and Munday, R. (1999) The Sensory Modality Assessment and Rehabilitation Technique (SMART): a comprehensive and integrated assessment and treatment protocol for the vegetative state and minimally responsive patient.
Neuropsychological Rehabilitation, 9(3/4): 305-320.
Gill-Thwaites, H. and Munday, R. (2004) The Sensory Modality Assessment and Rehabilitation Technique (SMART): a valid and reliable assessment for the vegetative state and minimally conscious patient.
Brain Injury, 18(12): 1255-1269.
Gill-Thwaites, H. (2006) Lotteries, loopholes and luck: misdiagnosis in the vegetative state patient.
Brain Injury, 20(13-14): 1321-1328.
Godbolt, A.K., Stenson, S., Winberg, M. and Tengvar, C. (2012) Disorders of consciousness: preliminary data supports added value of extended behavioural assessment.
Brain Injury, 26(2): 188-193.
Royal College of Physicians (2013) Prolonged disorders of consciousness, National clinical guidelines. .
London: Royal College of Physicians Publications.
Seel, R.T., Sherer, M., Whyte, J., Katz, D.I., Giacino, J.T., Rosenbaum, A.M., Hammond, F.M., Kalmar, K. and Pape, T.L. (2010) Assessment scales for disorders of consciousness: evidence-based recommendations for clinical practice and research.
Archives of Physical Medicine and Rehabilitation, 91(12): 1795-1813.
Shiel, A., Horn, S., Wilson, B.A., Watson, M.J., Campbell, M.J. and McLellan, D.L. (2000) The Wessex Head Injury Matrix (WHIM) Main Scale: a preliminary report on a scale to assess and monitor patient recovery after severe head injury.
Clinical Rehabilitation, 14: 408-416.
Tresch, D.D., Sims, F.H., Duthie, E.H., Goldstein, M.D. and Lane, P.S. (1991) Clinical characteristics of patients in the persistent vegetative state.
Archives of International Medicine, 151: 930-932.
Wade, D.T. (1996) Misdiagnosing the persistent vegetative state.
British Medical Journal, 313: 943-944.