Reliability and Validity of Diagnosis
Discuss the reliability and validity of Diagnosis There are many major systems of diagnoses worldwide, including the Diagnostic and Statistical Manual of Mental Health Disorders (DSM), International Classification of Diseases (ICD), and the Chinese Classification of Mental Disorders (CCMD). The DSM is one of the most widely used systems of diagnosis, and has undergone several editions. It is currently in its fourth edition. Diagnosis systems function to categorize and diagnose patients with mental health disorders such as depression, anorexia, schizophrenia and the like.
It provides a base for psychiatrists for a diagnosis to be given by listing the symptoms required for specific mental disorders. However, questions have been raised as to whether or not such systems of diagnosis are reliable or valid. The issue of reliability is one which looks at the question of consistency in diagnosis. Does the DSM or other diagnosis systems provide a consistent guide for psychiatrists to determine psychiatric disorders?
There are two types of tests that can be used to gage the reliability of diagnosis systems, inter-rater reliability and test-retest. Inter-rater reliability is the process where two different psychiatrists must give the same diagnosis to the same patient independently of each other, based on the symptoms that they have been presented with using the same classification guide. If the psychiatrists make the same diagnosis, the system is reliable. Test-retest refers to the same patient being given the same diagnosis at different times by the same psychiatrist.
Although the DSM is considered the most famous system of diagnosis, many studies have suggested that it is an unreliable system for diagnosing mental disorders, particularly in the earlier editions of the DSM. For example, Schmidt and Fonda (1956) asked two psychiatrists to diagnose independently 426 patients. There was great variety between the psychiatrists, especially for schizophrenia. (Inter rather reliability was used. ) The reliability of diagnosis also varies according to the particular mental disorders that are being observed in patients.
Kendall (1974) studied 1913 patients admitted to hospital since 1964, and then readmitted after 1969 (test-retest method). Research showed that schizophrenia was more often rediagnosed as a form of depression than the other way around. There was 70 percent stability in the diagnosis of depression, schizophrenia, dementia and alcoholism but less than 50 percent stability in the diagnosis of anxiety states, paranoid states, and personality disorders.
However, this can be argued that the less stable diagnoses can be caused by the subjectivity of anxiety, mainly because it is difficult to assess just how excessive a person’s worries are to be considered a mental disorder. Similarly, a study conducted by Di Nardo et al (1993) studied the reliability of the DSM III for anxiety seeking disorders. Two clinicians separately diagnosed 267 individuals seeking treatment for anxiety and stress disorders. They found high reliability for OCD but very low reliability for assessing generalized anxiety disorder.
Reliability of diagnosis also extends towards cultural differences. Merely by the different diagnosis systems available, such as the Chinese Classification of Mental Disorders (CCMD), cultural bound disorders are considered to exist. Some disorders identified in the ICD-10 and DSM-IV that are not common in China are left out of the CCMD, and others that are included in the CCMB appear to be culture-bond disorders found only within Chinese culture. Even among American and British culture, mental disorders are differently diagnosed.
In a study conducted by Cooper et al (1972), American and British psychiatrists watched clinical interviews and were asked to made diagnoses. American psychiatrists diagnosed schizophrenia twice as often, which British psychiatrists diagnosed mania and depression twice as often, thus showing a breach in cultural interpretations of mental disorders. The DSM is also considered unreliable for determining mental disorders according to different ages. Nicholls et al (2000) asked two practitioners to use either the DSM-IV, ICD-10 or Great Osmond Street Hospital’s own diagnostic system (GOS) to diagnose 81 children with eating problems.
It was found that inter rater reliability ranked low for the ICD and DSM, but the GOS system was more reliable than others, because it was specifically targeted for young children. However, explanations for the low reliability of diagnostic systems must take into account the different editions of the particular systems that were being used at the time. Later editions of the DSM have been found to be more reliable, as structured flow charts (decision trees) have been introduced to make psychiatrists diagnosis more orderly, as opposed to unstructured interviews at different times.
Demand characteristics in patients would have also played a part in affecting the different diagnoses by different psychiatrists in inter-rater reliability tests, as well as test-retest methods. Once a patient has been given a diagnosis, their behavior also often conforms to that of the diagnosis which they have been given. Because patients with the same disorder may have different symptoms, different diagnosis could also be easily given. The reliability of the DSM has, however been argued and supported by studies such as Goldstein in 1998.
Using the DSM III, Goldstein studied the effect of gender in schizophrenia. She rediagnosed 199 patients and found some differences. When two other experts were told to rediagnose a random sample of 8 of the patients using the case histories with all the indication of previous diagnoses removed, high levels of agreement and consistency were found in those diagnoses. When looking at diagnosis, the validity of diagnosis systems should also be considered apart from reliability. Validity of diagnostic systems focus more on the accuracy of diagnosis and the ability of a test to measure what it was designed to measure.
The key concern for diagnostic systems is whether they correctly diagnose people who really have particular disorders and do not give diagnosis to people who do not. This is difficult because in most disorders, there are no absolute standards against which people can be assessed and the diagnosis compared to. In other words, there is no guarantee that a patient has received a correct diagnosis. Tyrer et al (1988) studied the validity of the DSM III through a study on 201 patients with anxiety disorders. He found that there were many symptoms among anxiety disorders that overlapped with each other.
For example, similar symptoms existed between phobias, OCD, depression and eating disorders. However, because no diagnosis is clear cut, these disorders are likely to have similar problems and overlapping symptoms. Rosenham’s study in 1973 was an important study which challenged the validity of mental diagnosis systems. 8 sane people (pseudo-patients) were assigned to different mental hospitals across the USA. They were given perfectly normal histories, but were told to only display one symptom that could be interpreted as a mental health disorder.
They were told to tell the psychiatrists that they heard voices in their head saying ‘Empty, Hollow, Thud. ’ Despite the fact that this symptom could be interpreted as existentialist thinking, which was popularizing at that time, all pseudo-patients were admitted into the psychiatric hospitals with diagnosis of schizophrenia. Once in the hospital, the pseudo-patients were no longer required to display the ‘symptoms’ that landed them in the hospital. Instead, their goal was to get out.
No freedom was given to them however, and their sanity was not detected by any of the nurses or psychiatrists, but only by the other psychiatric patients. Upon discharge, all were diagnosed with schizophrenia in ‘remission. ’ Rosenham’s study had strong implications on the validity of diagnosis systems. It supported the idea that diagnosis systems could not be valid if psychiatrists were not even able to differentiate sane patients from those who were truly mentally ill. The inaccurate and false diagnoses that were in line with the issue of validity.
However, psychiatrists have defended the systems by arguing that as psychiatric diagnosis relies largely on the patient’s report of their experiences, faking their presence no more demonstrates problems with psychiatric diagnosis than lying about other medical symptoms. On the other hand, if that is the case, even more criticisms can be discussed about the validity and nature of diagnosis systems since self-reports by patients are often inaccurate as patients will give socially desirable answers and paint themselves in a better light.
Accurate, self-critical and honest answers are rarely exact in such cases. It is also important to note that the study was conducted during the time that the DSM II was still in use, hence it was not as well developed as current DSM editions. Later versions now in use, such as the DSM IV, are much more stringent and specific in their criteria. Through flow charts and structured interviews, psychiatrists are now more easily able to identify the distinctions between mental health disorders and accurately diagnose such cases. Social constructivists such as R.
D. Laing and Thomas Szasz take on an alternative view that also challenges the validity of diagnosis systems such as the DSM. Both uphold the argument that mental disorders are not biological realities and therefore cannot exist. Laing’s work, for example, suggests that although diagnosis is made within a medical model, the diagnosis is more of a social fact rather than a medical one. There are no reliable biological tests for diagnosing most psychological disorders, only guidance about categorizing behavior, thoughts and emotions.
According to social constructivists, we can only infer from people’s behavior that something is wrong, but we know that behavior is subject to cultural and social norms. Therefore, interpretation of behavior is subjective. Szasz also suggested that it is wrong to use a mental illness metaphor to describe behavior that does not conform to our expectations. Thus, if mental illnesses are merely results of social and cultural norms as argued by social constructivists, the DSM is seen as a system that is trying to describe illnesses that don’t objectively exist. If that is so, then it cannot be considered as something that is valid.
