Abstract
Several years ago, Al Amal Hospital began to use the Addiction Severity Index (ASI) for the assessment and measurement of the outcome of care provided for persons with addiction problems. Clinical staff have been sceptical about the usefulness of this tool in clinical practice. We conducted a survey of staff to record their opinions of the ASI and other outcome measurement tools. Participants highlighted factors that would encourage the use of such tools in daily practice. In order to achieve the desired goals, an outcome measurement tool should be concise, adapted to culture and suitable for use in treatment planning and follow-up by clinicians.
Outcome measurement has recently received considerable attention as part of the effort to improve the quality and effectiveness of healthcare (Duncan & Murray, 2012). The economic challenges currently faced by many countries around the world have resulted in the imposition of restrictions on health services. Clinicians are required to show the impact of their interventions. Other reasons for promoting the use of outcome measurements in daily practice include: improving quality of care, conducting effective treatment planning, assessing the cost-effectiveness of care (Holcomb et al, 1997), increasing the efficiency of services, communicating progress to patients and carers, and eliminating variations in clinical practice (Battaglia, 2001; Gilbody et al, 2002a, b; Holloway, 2002).
Outcomes mean different things to different individuals, depending on whether they are a care provider, a care receiver, a policy maker or a researcher. Clinicians are generally interested in clinical results (Battaglia, 2001); nevertheless, several studies have shown little evidence of routine outcome measurement in mental health practice. Other studies have found considerable ambivalence towards outcome measurement among psychiatrists and allied health professionals (Smith et al, 1997; Gilbody et al, 2002a, b; Duncan & Murray, 2012).
This study explored the opinions of clinical staff at Al Amal Hospital regarding the use of treatment outcome measures in addiction services. The study aimed to find out, from front-line staff, which factors facilitate the use of such tools in clinical settings. We thought the experience clinical staff have had with the ASI would be valuable in predicting their attitude towards the future use of outcome measurement in this hospital.
Method
A qualitative design was used to study the experience of clinical staff in the drug dependency unit at Al Amal Hospital in Riyadh. The inclusion criteria were: being a member of one of the treatment teams on the unit, plus having taken the role of care coordinator. All care coordinators were included in this survey, which took place in February 2013. Care coordinators are responsible for completing the ASI; most of them have received training on its use. We used a questionnaire to explore the views of the target group of their experience with the ASI and to record their opinions on the measurement of outcome of care in addiction services.
The study was approved by the hospital’s research committee. Tables were used to summarise and code data. Key themes were derived from care coordinators and then compared and contrasted. Transcripts were rated by two independent researchers, as were the data derived for coding, and consensus was reached.
Results
The questionnaire was completed by 24 of the 31 clinicians who fulfilled the inclusion criteria for this study (Table 1), a 77% response rate.
Table 1. Questionnaire responses.
| Profession | Number | Responded |
|---|---|---|
| Consultants | 3 | 2 |
| Specialists | 7 | 6 |
| Residents | 8 | 8 |
| Social workers | 5 | 3 |
| Clinical psychologists | 8 | 5 |
| Total | 31 | 24 (77%) |
Open in a new tab
Experience with the ASI
The clinicians could be placed in two groups regarding their opinions on the use of the ASI. One group described the ASI as a tool that is useful for collecting comprehensive information about patients, but nevertheless thought it did not have an impact on the quality of patient care. The other group expressed a very negative view of the ASI, citing the complexity of the scale and, consequently, the increase in workload.
Strengths of the ASI
There was agreement among the participants that the greatest strength of the ASI is its ability to allow the collection of detailed information about patients’ drug problems as well as medical, psychological and social history. In addition, specialist and resident doctors believed that, if used properly, the ASI is useful in developing a person-centred treatment plan that includes relapse prevention. Few thought the ASI would be useful for research, however.
Limitations of the ASI
The excessive length and the time-consuming nature of the ASI were its main drawbacks, as identified by almost all clinicians. Other concerns included lack of sensitivity to cultural differences; respondents believed the questions had not been properly adapted to Saudi culture. Moreover, they believed that, as the ASI was not used in follow-up, there was no proper measurement of the outcome of treatment. Inadequate training sufficed to throw into doubt the usefulness of this scale in the way it was used.
General use of outcome measurements in psychiatric and addiction services
Some clinicians did not believe that the use of an outcome measurement scale would add value to patient care, but others did believe such a scale would be valuable in treatment planning and follow-up.
Desirable characteristics of an outcome measurement scale
The clinicians indicated that an outcome measurement scale should be concise, require only a short time to complete and be easy to use. They believed these characteristics would encourage its use in daily practice. Adaptation of the scale to the local culture, sensitivity to change in patient presentation and ease of interpretation were other desirable characteristics identified by the group.
Discussion
This work helps to clarify why the use of the ASI has not achieved the goals intended, mainly measuring the outcome of care, at this Saudi hospital. In addition, it proposes methods that can be used to render an outcome measurement scale more useful within clinical settings. This is important, especially with the current drive for using outcome measurement tools in Saudi hospitals.
The achievement of better treatment outcomes and the provision of high-quality care that produces a measurable improvement in the patient’s clinical state were clearly desirable to all the members of staff surveyed. Medical and nonmedical staff believed that the use of care outcome measurement tools would improve the quality of care. The findings also highlighted the desirable characteristics of an outcome measurement tool, from the view point of staff.
The study showed that the experience with the ASI had not been as successful as anticipated, for several reasons. These included the length of the tool and its impact on workload. Further, the information obtained from the ASI was not used properly in establishing management plans and neither in follow-up nor research. This seems to have led to a loss of the enthusiasm with which the ASI was first met. Insufficient training probably affected the reliability of this tool. There is a belief that the translation of the ASI did not take into consideration cultural differences, which would potentially reduce the validity and relevance of information obtained from Saudi patients.
Although the findings support the use of outcome measurement tools at this Saudi hospital, they nevertheless raise concerns that a new outcome measurement tool might be used inconsistently by staff unless the hospital addresses the issues raised in the survey. These issues are consistent with the available evidence on what is considered necessary to render an outcome measurement tool suitable for use in daily practice (Slade et al, 1999; Battaglia, 2001; Zimmerman et al, 2008; Duncan & Murray, 2012).
Conclusions
Despite being available for several years, the ASI has not been properly used in this hospital. There has been no research into this or the impact on staff workload and patient care. We found evidence from staff that the introduction of the ASI had not provided significant benefits to patients or clinicians. Careful consideration should be taken of the views expressed by clinical staff if the use of an outcome measurement tool is to achieve the desired goals. A locally developed tool that takes into consideration cultural factors would be more useful in determining future policies and impacts.
Limitations
This was a qualitative, not a quantitative study. As such, rich and detailed accounts were acquired from participants, but the group declining to participate might have had different opinions.
References
- Battaglia, J. (2001) Medscape education. Available at http://www.medscape.org/viewarticle/418603 (accessed June 2013).
- Duncan, E. A. S. & Murray, J. (2012) The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Services Research, 12, 96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gilbody, S. M., House, A. O. & Sheldon, T. A. (2002a) Outcomes research in mental health: systematic review. British Journal of Psychiatry, 181, 8–16. [DOI] [PubMed] [Google Scholar]
- Gilbody, S. M., House, A. O. & Sheldon, T. A. (2002b) Psychiatrists in the UK do not use outcomes measures: national survey. British Journal of Psychiatry, 180, 101–103. [DOI] [PubMed] [Google Scholar]
- Holcomb, W. R., Parker, J. C. & Leong, G. B. (1997) Outcomes of inpatients treated on a VA psychiatric unit and a substance abuse treatment unit. Psychiatric Services, 48, 699–704. [DOI] [PubMed] [Google Scholar]
- Holloway, F. (2002) Outcome measurement in mental health: welcome to the revolution. British Journal of Psychiatry, 181, 1–2. [DOI] [PubMed] [Google Scholar]
- Slade, M., Beck, A., Bindman, J., et al. (1999) Routine clinical outcome measures for patients with severe mental illness: CANSAS and HoNOS. British Journal of Psychiatry, 174, 404–408. [DOI] [PubMed] [Google Scholar]
- Smith, G. R., Fischer, E. P., Nordquist, C. R., et al. (1997) Implementing outcomes management systems in mental health settings. Psychiatric Services, 48, 364–368. [DOI] [PubMed] [Google Scholar]
- Zimmerman, M., Chelminski, I. & McGlinchey, J. B. (2008) A clinically useful depression outcome scale. Comprehensive Psychiatry, 49, 131–140. [DOI] [PubMed] [Google Scholar]
