Toggle Main Menu Toggle Search

Open Access padlockePrints

Ensuring treatment fidelity in intervention studies: Developing a checklist and scoring system within a behaviour change paradigm

Lookup NU author(s): Jo Baker, Professor Helen StringerORCiD, Professor Cristina McKeanORCiD

Downloads


Licence

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).


Abstract

© 2023 The Authors. International Journal of Language & Communication Disorders published by John Wiley & Sons Ltd on behalf of Royal College of Speech and Language Therapists. Background: Treatment fidelity refers to the degree to which an intervention is implemented as intended. Promoting treatment fidelity is important to achieve a valid comparison in intervention research. However, it is often underreported: few studies detail the use and development of fidelity measures. This study aims to promote the treatment fidelity of a modified version of the Derbyshire Language Scheme (M-DLS), a manualised intervention for children with language difficulties, by exploring participants’ opinions on training and intervention delivery. Results inform development of a checklist and scoring system to monitor and promote treatment fidelity in a comparison trial. Method: Ten student speech and language therapists (SLTs) and two research assistants (RAs) participated in the study. All received training on the M-DLS, and 10 were video-recorded completing role-plays of an M-DLS session in small groups. Feedback was gathered after training and role-plays in focus groups and interviews. Feedback was interpreted using the constructs of the Theoretical Domains Framework (TDF). A treatment fidelity checklist was then developed using the feedback. The first author and two RAs rated role-play videos using the checklist to trial it to inform amendments and to promote interrater reliability. Interrater agreement was calculated using Spearman's test of correlation. Results: Participants discussed the importance of having clear materials and time to practise sessions. They suggested amendments to the materials and training to promote treatment fidelity. The checklist and scoring system accounted for participants’ suggestions, with amendments detailed in a log. Spearman's correlation results suggested agreement between the raters was strong. Conclusions: Results emphasise the importance of training quality, practice and reflective opportunities and clear materials to promote treatment fidelity. The construction of the checklist and scoring system was described in detail, informing the development of future checklists. After further trialling, the checklist can be used to ensure the M-DLS is delivered with high treatment fidelity in the comparison trial. What this paper adds: What is already known on this subject Treatment fidelity is an essential component of intervention effectiveness and efficacy studies, ensuring the intervention is delivered as intended. It is also an essential component of evidence-based clinical practice. However, few research studies report the treatment fidelity process or publish the checklists used, depriving clinicians of useful information for implementation. What this study adds This study describes in detail the iterative process of treatment fidelity checklist development, engaging those implementing the intervention in development. This ensured clarity and interrater reliability of the checklist. Furthermore, a novel scoring system was developed so that accuracy of implementation can be easily compared across users and across practice attempts. What are the clinical implications of this work? The importance of treatment fidelity when implementing effective and efficacious interventions cannot be overstated. The treatment fidelity checklist developed for research can be easily adopted to support accurate implementation in clinical practice through an audit process.


Publication metadata

Author(s): Baker J, Stringer H, McKean C

Publication type: Article

Publication status: Published

Journal: International Journal of Language and Communication Disorders

Year: 2024

Volume: 59

Issue: 1

Pages: 379-395

Print publication date: 01/01/2024

Online publication date: 16/09/2023

Acceptance date: 28/08/2023

Date deposited: 03/10/2023

ISSN (print): 1368-2822

ISSN (electronic): 1460-6984

Publisher: John Wiley and Sons Inc.

URL: https://doi.org/10.1111/1460-6984.12955

DOI: 10.1111/1460-6984.12955

Data Access Statement: The data that support the findings of this study are not available due to privacy restrictions.


Altmetrics

Altmetrics provided by Altmetric


Funding

Funder referenceFunder name
Heather van der Lely Foundation

Share