ChatGPT's Ability to Assist with Clinical Documentation: A Randomized Controlled Trial

Abstract

Introduction: Clinical documentation is a critical aspect of health care that enables healthcare providers to communicate effectively with each other and maintain accurate patient care records. Artificial intelligence tools, such as chatbots and virtual assistants, have the potential to assist healthcare providers in clinical documentation. ChatGPT is an artificial intelligence conversational model that generates human-like responses to text-based prompts. In this study, we sought to investigate ChatGPT's ability to assist with writing a history of present illness based on standardized patient histories.

Methods: A blinded, randomized controlled study was conducted to compare the use of typing, dictation, and ChatGPT as tools to document history of present illness (HPI) of standardized patient histories. Eleven study participants, consisting of medical students, orthopaedic surgery residents, and attending surgeons, completed three HPIs using a different documentation technique for each one. Participants were randomized into cohorts based on the type of documentation technique. Participants were asked to interview standardized patients and document the patient's history of present illness using their assigned method.

Results: ChatGPT was found to be intermediate for speed; dictation was fastest, but produced markedly longer and higher quality patient histories based on Physician Documentation Quality Instrument score compared with dictation and typing. However, ChatGPT included erroneous information in 36% of the documents. Poor agreement existed on the quality of patient histories between reviewers.

Discussion: Our study suggests that ChatGPT has the potential to improve clinical documentation by producing more comprehensive and organized HPIs. ChatGPT can generate longer and more detailed documentation compared with typing or dictation documentation methods. However, additional studies are needed to investigate and address concerns regarding privacy, bias, and accuracy of information.