Salesforce logo

Redesign of the data prediction tool to guide and support low AI maturity customers

UI/UX

B2B/SaaS

AI/ML

Role

UX Design Extern

Duration

Aug-Dec 2022

Team

3 UX Designers, 1 Product Manager, 1 Data Scientist, EPB product team

Context

EPB: AI tool for predicting future business outcomes

The Einstein Prediction Builder (EPB) allows customers to make predictions using their own data without coding. For example, restaurant managers can use EPB to predict which customers are likely to show up for reservations so they can properly manage staffing.

Target Users

Salesforce admins with low AI maturity

Business Analysts
Marketing Professionals
Sales Managers
  • Familiar with Salesforce
  • New to EPB
  • Know very little about AI/ML

Problem

As a low-code tool, EPB is not easy to use

Customers have mentioned that EPB is not serving their needs. Even though the product allows users to create experiences without coding, AI is a complicated process and customers need more help.

Problem: low adoption rate, low satisfaction rate, low model deployment rate, and low design system utilization

Business Ask

Increasing EPB adoption by low AI maturity customers

Discover

Identifying user drop-off with data scientists

To kick off the project, we worked closely with data scientists to understand the user flow and found that there was significant user drop-off during the prediction building process.

User flow of EPB, showing a large drop-off of user numbers in the prediction building process

Research

Digging into the problem through user research

Next, we conducted extensive research to dig deeper into the problem and identify more pain points and opportunities.

Research: heuristic evaluation
Research: competitive analysis
Research: user testing

Current Situation

Dashboard gaps & Underutilized data checker

EPB uses a data checker to help users assess the quality of data throughout the processes. However, we found in testing that users tend to ignore it due to its limited functionality and unintuitive design.

A screenshot of current EPB interface

Synthesis

Translating research insights into design strategies

By recognizing existing issues, we synthesized all our findings into three high-level insights and identified three design strategies.

Insight#1: inefficient to useStrategy#1: enhance user guidance
Insight#2: High learning curveStrategy#2: Improve user understanding
Insight#3: Overlooked data checkerStrategy#3: Interactive data checker

Crafting Design

How might we improve the prediction-building process?

Brainstorming

Ideating for complex product through bodystorming

Given the complexity of the product, we found it challenging to come up with ideas using traditional brainstorming. So instead, we fully immersed ourselves in the product's context, wrote down ideas as we went through it, and grouped them into main concepts.

An affinity diagram of ideas generated in bodystorming.

Concept Evaluation

Prioritizing concepts with stakeholders

With the help of the Salesforce Design Team, we evaluated our concepts on the Value/Effort Matrix with a scale of (0-5). We then moved forward with three most valuable and feasible concepts.

Prioritization of concepts based on value/effort matrix.

Concepts

Streamlined dashboard & Engaging data interaction

The final concepts include two aspects: a streamlined dashboard that simplifies navigation and visualizes progress, and a redesigned data checker that provides engaging data interactions.

Concepts: #1 Streamlined dashboard; #2 Engaging data interaction.

Ideation

Differentiating data visuals for better understanding

We found through our research that users struggle with complex data sets. To solve this problem, the new data checker uses colors to mark different types of data for easier understanding.

Sketch of a new data checker that color-codes different types of data.

Remapping Information Architecture

Finding a home for the new data checker

Incorporating the new data checker, we explored three design options and chose the "Popover" approach for the best user experience.

Wireframes of the dashboard design, exploring three different options to incorporate new data checker.

Solution

An interactive data checker that guides and supports users during the process

Feature#1

Revamped dashboard layout

We revamped the information architecture of the dashboard by simplifying navigation, visualizing progress, and making the data checker more visible and interactive.

New dashboard design of EPB: simplified navigation, visualized progress, and interactive data checker.

Feature#2

Data status at a glance

With the new data checker, users can visually see the different types of data as well as the changes during configuration. Interacting with it also helps them better understand the dataset structure.

A GIF showing how the interactive data checker works
A data visualization scheme of 6 steps in prediction building

Feature#3

Recognize, diagnose, and recover from errors

Moreover, users will be alerted when an error occurs, and they can easily fix it by following the suggestions and shortcuts provided by the data checker.

A GIF showing how the data checker help recover from errors

Iterations

More efficient, consistent, and user-friendly

How to Iterate

Engage product teams for continuous improvement

Without access to users, feedback from EPB design experts was critical. Through weekly meetings, I pinpointed areas for improvement and ensured that the design was better aligned with end users.

Iteration#1

Streamlined Data Checker Popover

The popover allows the user to quickly review the settings and jump to steps when needed.

Before & after comparison of the data checker popover window when an error occurs

Iteration#2

Intuitive Navigation Bar Redesign

With these changes, users can efficiently move between different sections, making the overall experience smoother and more intuitive.

Before & after comparisons of the navigation bar

Iteration#3

Progress Indicator

A place to show current progress and remaining steps.

Before & after comparison of the progress indicator

Evaluation

Measuring success by A/B testing with users

Participants

20

Low AI maturity users w/o prior experience with EPB

Test Environment

Remote, unmoderated testing

Test Scenarios

Participants were asked to complete a set of tasks

Using both the original and the redesigned versions

Metrics to Measure

We measured our design by these metrics:

Prediction building efficiency

To measures the speed and accuracy of the user when building predictions.

Error recovery rate

To measures the user's ability to recover from errors during the process.

Prediction completion rate

The percentage of completed predictions out of all attempts.

User satisfaction

The level of satisfaction of the user after using the Einstein prediction builder.

Impact

We saw overall improvement in usability and efficiency!

Learning

๐Ÿ† Entering unknown domains: dealing with complexity

Data and machine learning was an unfamiliar field for me. When I joined the project, I had no idea how the tools worked or even what many of the terms meant. However, by constantly asking questions, learning the ropes quickly, working closely with the technicians, and actively listening to stakeholders, we were able to impress our manager with what we accomplished on this project.

Learning

๐Ÿ„๐Ÿปโ€โ™€๏ธ Playing with design systems: constraints or shortcuts?

While we initially felt constrained, a mature design system like Salesforce Lighting brought consistency and efficiency to our design. I had to really delve into its principles, components, and guidelines and think hard about striking a balance between following and adapting. I also learned to push the boundaries of design systems when appropriate.

Thanks for reading! ๐Ÿ™‡๐Ÿป