Case Study:

Making Ceba smarter

Role

UX Designer

Duration

6 months

Overview

A multi feature project uplifting our AI chatbot processes to enable chat containment.

UX Discovery
UI Design
UX Research
THE PROBLEM SPACE
Current state of disputes (2023)

Our AI chatbot Ceba enables customers to self serve or send through information for processing requests

In order for the agents to process these requests, the information that Ceba captures from the customers must be accurate. However customers are able to free text anything they want within Ceba, and Ceba simply accepts it and moves on to the next question. For customers trying to submit a dispute, we’re seeing a 13% drop out rate when asked about account details, and a 30% drop out rate when asked about transactions.

The current process is ineffective in containing customers within the flow or are often unable to be processed due to incomplete information from customers.

Key problems

1. Customers have to leave Ceba in order to find or edit account information

In several touchpoints, customers are needing to exit Ceba in order to do tasks such as updating their address or finding their card number. We know that these are high drop out points, with many customers not returning after leaving.

2. Customers have to manually input their details

Customers are asked to input information such as account number, transaction details and contact details which is tedious and difficult. Furthermore there is no validation of customer input, leading to customers typing in anything in order to get through to a human agent.

3. There is no ability to review and edit their answers

Customers can’t go back to a previous question if they make a typo or enter the wrong details, causing them to abandon and have to restart the flow in order to change their answers.

Not only are these known customer pain points from captured feedback. We also know that this is a huge business problem as customers abandoning these flows or submitting inaccurate information leads to increase in frontline agent work.

CHALLENGE

Personalising Ceba through APIs ✨

The proposed business solution is to implement customer data APIs so that we can display customer data and pull through account and transaction details so that customer’s can select from rather than manually inputting them.

The UX Challenge

From a technical perspective, we can use APIs to pull in customer information. How do we do this gracefully and ensure all customer scenarios are catered for? Key challenges I had to think about include:

  • Chatbots are not designed for complex data inputs, the current platform has it’s limitations in what we can achieve

  • You can’t simply backtrack in a conversation, unlike forms

  • APIs are great but they can fail, we need to handle errors with care

LOFI EXPLORATION - INTERACTION MODELS

Designing the interaction pattern to set the foundation for future data input

Although our first initiative looked at implementing an account selector. As a UX activity I needed to create a pattern that can be reused for any future data capture and display within the chat. After some research and exploration, few interaction models were created to get feedback on, I explored having in line interactions and cards, some interaction patterns explored are:

1. Customers select and confirm

Customers have to select and then tap confirm in order to submit. The benefits of this being less room for error, and is the preferred design.

2. Customers select without confirming

Customers tapping on an account will immediately select it as the chosen, likely more errors however requires less tap from customers and they are still given the option to correct it at a later step. However, this pattern will only work for single select and there is possibility in enabling multiselect in the future. 

TECH & BUSINESS ALIGNMENT

Critical factor to success is aligning with business & tech early to understand feasibility

I designed and facilitated a tech alignment workshop with our whole squad including our developers, testers, BAs, PO to gain a thorough understanding of:

  • What data the API delivers

  • Possible error scenarios

  • Areas for further investigation

  • Align on initial design & requirements

SCREEN FLOWS

Understanding all possible scenarios to paint the bigger picture

Following the workshop, I was able to map out, iterate and validate the screen flow with the developers to illustrate and guide the developers and BA on what happens at each step of the interaction and possible scenarios.

This challenged me to think critically on the types of errors that can occur and cater for them, as initially I naively thought there would only be one API error, I learnt that there was an API for the error scenario too! I worked with copy and the conversation designers as well to best design a flow to inform the users what is happening and allow them to move forward. 

SOLUTION

The Account Selector

The account selector allows customers to easily select from their list of accounts. Although simple, this set the foundational pattern for future API and structured data inputs within the chatbot.

The Transaction Selector

The transaction selector was a more complex piece of work. Due to the large number of transactions and the ability to multi select. Design considerations included:

  1. Allowing users to review and confirm
    With the ability to multiselect and a long list of transactions, it is likely that users may select the wrong transactions. By providing a second step to review transactions, we mitigate some risk of users selecting the wrong transactions

  2. Accessibility of a long list
    I consulted the accessibility team early to understand the implications and risk of the long list of transactions, particularly on voiceover capabilities. These were documented early for the developers to consider when building to ensure it is accessible for all

  3. Ability to filter
    Due to the long list, I explored different options in ways to filter and search for transactions. However due to time restraints, we’ve set out the scope for the MVP to not include any filtering. This is revisited post the release of the MVP. 

USABILITY TESTING

Users found it so straight forward, they didn’t even mention it

I took the both the account and transaction selectors to 1:1 Usability Testing Sessions in collaboration with our UX researchers to test the interaction patterns.

  • The displayed information was deemed sufficient to help participants select the right account

  • Participants were quick to move forward with the task and tap on ‘Select transaction’. All participants appeared satisfied to move on to the next step with no trouble.

  • All participants were able to scroll to locate the transaction of interest.

What worked well:

  • One participant did not notice the button at the bottom

What didn’t work well: 

OUTCOMES & NEXT STEPS

We saw a 20% increase in containment rates since release

With the API uplifts and new UI interactions, users no longer need to jump out of the flow to find details. We found that this increased containment rates by 20%. This work also won our squad the quarterly CEO excellence award!

There’s still work to do…

Although this is a significant improvement to the customer experience. Our metrics also tell us that there has been a small spike in customers timing out at the transaction selector step.

Data driven insights

To tackle this spike in the timeout, our squad kicked off an analysis to add a filtering option into the transaction selector. The assumption is that customers are not finding the transaction they are looking for and thus timing out.

I took on the task to do some analysis with customer feedback, conversation transcripts along with our conversation designer and found that customers are often trying to dispute more than 10 transactions along with many more pain points, that could be the cause of the timeout. Due to a restructure, this work needed to be handed over to a different squad. Thus I created a current state service map of the disputes process, outlining the pain points and areas of opportunity.

Other case studies

Unified Messaging

Helping customers find their messages quicker.

Consulting costing tool

Design of a new web tool used by all EY people globally.