Background: Using artificial intelligence approaches enable automated assessment and analysis of speech biomarkers for Alzheimer's disease, for example using chatbot technology. However, current chatbots often are unsuitable for people with cognitive impairment. Here, we implemented a user-centred-design approach to evaluate and improve usability of a chatbot system for automated speech assessments for people with preclinical, prodromal and early dementia.
Method: Participants (n = 221 by January 2024, 300 planned, see Figure 1) of the PROSPECT-AD study were automatically called for six times every three months by our chatbot "Mili". In each call three cognitive tasks (Wordlist, Semantic Verbal Fluency, Story Telling) are offered. We applied the System Usability Scale (SUS) in all participants and conducted additional semi-structured interviews in a subgroup to gain in-depth information for improving Mili and adapting it to the users' needs. During the interviews, we focus on affinity for technology and Mili's usability.
Result: We collected usability data at M3 by the SUS from 163 participants (12 missings, ØAge: 73.2 years, SD: 6.6 years, see Figure 2) after their second call. The results indicated a good usability of Mili (ØSUS: 70.4/100, SD: 22.2). Within semi-structured interviews we asked 22 participants (ØAge: 73.0 years, 12 female, ØMMST: 29.3, range 24-30, medium tech-savvy) for their ideas to improve the Mili system. We analysed the interviews using qualitative content analysis. Participants wished for an intelligent chatbot and a feedback system after each call about participants' cognitive performance. In the event that the performance would indicate cognitive decline, Mili should provide information on local services for dementia, and give recommendations for prevention and advises for the user's physician for medication. Seventeen interviewees (2 undecided, 3 negative) would like to use Mili regularly if they would receive a feedback on their cognitive performance.
Conclusion: Our results revealed a high usability and feasibility of the automated phone calls by Mili. Eligible feedback solutions would increase the usefulness and usage of remote testing if a they provide individual advises for prevention and contact persons. Unrealistic expectations on Mili regarding the recommendation of medication must be discussed with the users.
© 2024 The Alzheimer's Association. Alzheimer's & Dementia published by Wiley Periodicals LLC on behalf of Alzheimer's Association.