Open E-survey on the Use and Perception of Chatbots in Vascular Surgery

Fabien Lareyre; Mario D'Oria; Caroline Caradu; Vincent Jongkind; Gilles Di Lorenzo; Matthew R Smeds; Bahaa Nasr; Juliette Raffort; Study group collaborators: European Vascular Research Collaborative (EVRC)

doi:10.1016/j.ejvsvf.2024.07.037

Open E-survey on the Use and Perception of Chatbots in Vascular Surgery

EJVES Vasc Forum. 2024 Jul 16:62:57-63. doi: 10.1016/j.ejvsvf.2024.07.037. eCollection 2024.

Authors

Collaborators

Study group collaborators: European Vascular Research Collaborative (EVRC):
Florian Enzmann, Gert J de Borst, Joel Ferreira Sousa, Lewis Meecham, Liliana Domingos, Martin Teraa, Petar Zlatanovic, Salome Weiss, Stefano Ancetti, Albert Busch, Bergrós Jóhannesdóttir, Alexander Gombert, Katariina Noronen, Robert Hinchliffe, Alexandru Predenciuc, Panagiotis Doukas, Leszek Kukulski, Qasam Ghulam, Angelos Karelis, Maram Darwish, Mohammad Esmaeil Barbati, Markvard Møller, Matt Spreadbury, Willemien van de Water, Desiree van den Hondel, Harm Ebben, Alexander Croo, Gilles Uijtterhaegen, Adina Trusca, Ryan Gouveia Melo, Vaiva Dabravolskaite, Paolo Spath, Vishal Amlani, Aoife Kiernan, Christian Zielasek

Affiliations

¹ Department of Vascular Surgery, Hospital of Antibes Juan-les-Pins, Antibes, France.
² Université Côte d'Azur, CNRS, UMR7370, LP2M, Nice, France.
³ Fédération Hospitalo-Universitaire FHU Plan&Go, Nice, France.
⁴ Division of Vascular and Endovascular Surgery, Cardiovascular Department, University Hospital of Trieste, Trieste, Italy.
⁵ Bordeaux University Hospital, Department of Vascular Surgery, Bordeaux, France.
⁶ Department of Surgery, Amsterdam UMC, location Vrije Universiteit, University of Amsterdam, Amsterdam, the Netherlands.
⁷ Division of Vascular and Endovascular Surgery, Department of Surgery, Saint Louis University, Saint Louis, MO, USA.
⁸ Department of Vascular and Endovascular Surgery, Brest University Hospital, Brest, France.
⁹ Institute 3IA Côte d'Azur, Université Côte d'Azur, France.
¹⁰ Clinical Chemistry Laboratory, University Hospital of Nice, Nice, France.

Abstract

Objective: Large language models and artificial intelligence (AI) based chatbots have brought new insights in healthcare, but they also raise major concerns. Their applications in vascular surgery have scarcely been investigated to date. This international survey aimed to evaluate the perceptions and feedback from vascular surgeons on the use of AI chatbots in vascular surgery.

Methods: This international open e-survey comprised 50 items that covered participant characteristics, their perceptions on the use of AI chatbots in vascular surgery, and their user experience. The study was designed in accordance with the Checklist for reporting Results of Internet E-Surveys and was critically reviewed and approved by international members of the European Vascular Research Collaborative (EVRC) prior to distribution. Participation was open to self reported health professionals specialised (or specialising) in vascular surgery, including residents or fellows.

Results: Of the 342 individuals who visited the survey page, 318 (93%) agreed to participate; 262 (82.4%) finished the survey and were included in the analysis. Most were consultants or attending physicians (64.1%), most declared not having any training or education related to AI in healthcare (221; 84.4%), and 198 (75.6%) rated their knowledge about the abilities of AI chatbots between average to very poor. Interestingly, 95 participants (36.3%) found that AI chatbots were very useful or somewhat useful in clinical practice at this stage and 229 (87.4%) agreed that they should be systematically validated prior to being used. Eighty participants (30.5%) had specifically tested it for questions related to clinical practice and 59 (73.8%) of them experienced issues or limitations.

Conclusion: This international survey provides an overview of perceptions of AI chatbots by vascular surgeons and highlights the need to improve knowledge and training of health professionals to better evaluate, define, and implement their use in vascular surgery.

Keywords: Artificial intelligence; Chatbot; Large language model; Natural language processing; Survey; Virtual assistant.