I’m a UX designer who identifies problems from the user’s perspective and creates solutions from multiple angles. With a people-first mindset, I am dedicated to bridging the gap between users and technology/services through proactive learning and collaboration.
Conversational breakdowns commonly happen in task-oriented banking chatbots, reducing user trust and long-term engagement. This study aimed to compare multi-modal (a new approach) and option-based (a common approach) repair strategies in banking chatbots to evaluate their effectiveness in supporting transparency, trust recovery, and continued use after conversational breakdowns.
In order to achieve this, it involved designing a multi-modal repair strategy through a user-informed design approach, systematic evaluation based on the UTAUT-AI model (Wanner et al., 2022), and providing implications for an effective repair strategy for a conversational breakdown, following these research questions:
Background
#Burden from self-repair
#Difficulty of human-AI conversation
Conversational breakdowns—which the chatbot misunderstands their intent or fails to process input (Følstad, Law and Van As, 2024) place the burden of repair entirely on the user. The difference in communication style between AI and humans leads to multiple repair trials, ultimately resulting in low trust and abandonment (Cai, Heo, and Yan, 2025; Ashktorab et al., 2019).
Current Repair Strategies
#lack of transparency, #Gulf of Execution & Evaluation
Option-based repair is a solution-based approach and favoured by users due to its minimal effort in reformulating their input. However, the lack of explanation creates gaps that correspond with the Gulf of Evaluation (lack of feedback) and the Gulf of Execution (uncertainty in how to fix input errors) (Whitenton, 2018).
Multi-modal
#Learning curve, #cognitive load
Multi-modal repair strategy by Li et al., (2020), showed the potential benefits to enhance transparency and users’ satisfaction. Yet, it introduced a learning curve due to unfamiliar interaction and process.
Link to Approach
As the previous study found the opportunity to deal with the limitation of current repair strategies, the study focused in developing a multi-modal repair strategy: RQ1) how to design a multi-modal repair strategy by minimising cognitive load while enhancing transparency
Multi-modal served as a treatment prototype, while option-based was a baseline prototype. According to the approach based on the defined problems, the multi-modal repair prototype was designed with users through a co-design workshop. Furthermore, both prototypes were implemented by Vibe coding with multiple AI tools to make them as real and interactive a chatbot as possible.
Design system for both prototypes
Option-based
As the baseline prototype, it reflected common practices in mobile banking chatbots in the UK and South Korea.
Multi-modal
The co-design workshop informed its design features, which minimise cognitive load from the user’s perspective.
The controlled experiment was conducted using a within-subjects design, with counterbalancing between the two prototypes. A total of 23 participants were recruited and asked to complete a task: “Finding out the interest rate for a loan based on your salary.” Based on the UTAUT-AI model, six constructs were measured using a 7-point Likert scale. Additionally, qualitative data was collected through semi-structured interviews conducted after the post-task questionnaire, to gain a richer and deeper understanding.
The multi-modal repair strategy achieved higher scores across all measures than the option-based approach, but these differences were not statistically significant due to wide individual variation in user responses. Qualitative insights revealed the trade-off: multi-modal repair enhanced transparency and supported trust-building(aligning with Do et al. (2024)), while option-based repair was valued for its familiarity and simplicity, which encouraged continued use after a breakdown
Multi-modal Repair is potentially effective for a conversation breakdown, but it shows adaptation effort as new challenge. It indicates the importance the trade off between enhanced transparency and adaptation effort for an effective repair strategy
1. Explicit and Actionable System Cues
Provide visual system status (e.g., highlights, inline tips) so users know exactly what went wrong and how to fix it. Clear system status reduces trial-and-error and builds trust.
2. Balance transparency with familiarity
Users often prefer familiar, low-effort flows even if less transparent. Therefore, a new approach must combine with familiar interaction practices to reduce adaptation cost.
3. Reduce cognitive overhead as per the mental model of the users
Guidance must align with users’ mental models, not the designer’s assumptions. Mismatched wording or unclear instructions increase unnecessary effort.
4. Plan adaptive onboarding for new interactions
Structured onboarding helps users build an accurate mental model and be confident to interact with new interactions/approaches.
