Project Excel
Heuristic Evaluation · User Testing
At the request of Rob Thomas, SVP of IBM Cloud and Data Platform, "Project Excel" was initiated to identify barriers and prototype solutions across the trial user experience for IBM's Cloud Paks — from discover and learn through register, get started, first use, and purchase.
01 // Overview
Why this study existed
IBM Cloud Pak for Business Automation is a set of integrated, market-leading software designed to help organizations solve their toughest operational challenges — with actionable AI-generated recommendations, built-in analytics to measure impact, and business-friendly low-code tooling.
For the Cloud Pak for Business Automation page, our task was to build the as-is user journey map for the Discover, Learn, Try, and Demo stages — then pressure-test that journey with a heuristic evaluation and moderated user testing to expose what was blocking trial conversion.
Objectives
- 01Identify barriers in the trial experience for IBM's Cloud Pak for Business Automation.
- 02Map the as-is user journey for the Discover, Learn, Try, and Demo stages.
- 03Surface usability issues using Nielsen's 10 heuristics.
- 04Recommend solutions that improve discoverability, navigation, and clarity.
02 // Method
Methodology
Heuristic Evaluation + User Testing
A combined heuristic walkthrough and moderated usability study of the Discover → Learn → Try → Buy journey. Participants worked through a realistic evaluation scenario for adopting a business automation platform.
- › 6 practitioners across operations, marketing, and analytics
- › Customer Ops Manager, Senior Manager, Marketing Ops, Business Analyst, System Admin
- › Scenario-based tasks across the marketing page and SaaS trial
- › Nielsen's 10 usability heuristics scored per surface
Stakeholders
Automation, marketing, analytics, SaaS, design, and research teams aligned on shared objectives. I led the research study end-to-end, partnering with design, analytics, and offering managers throughout.
"Imagine you want to evaluate a business automation platform to address workflow orchestration, decision management, and task automation — so you can build AI-directed workflows and gain better governance over your enterprise content."
03 // Issues
What we surfaced
The heuristic evaluation surfaced 42 issues across the Discover, Learn, Try, and Buy journey — concentrated heavily in the trial and getting-started experience.
Significant usability issues that are difficult to recover from. Fix immediately.
Frequent issues that impact usability and require user effort to resolve.
Small issues that still affect the experience and should be cleaned up.
| Stage | Critical | Moderate | Minor |
|---|---|---|---|
| Trial / Getting Started | 8 | 8 | 4 |
| Learn | 1 | 4 | 3 |
| Signup | 2 | 1 | 3 |
| Discover / Marketing | 2 | 3 | 3 |
04 // Summary
Six themes from the field
A need for guidance
Users wanted a thorough, step-by-step tour after entering the trial. The existing Walk Me tour felt shallow and left them unsure of the next action.
Navigation issues
Links inside the trial pushed users out of the product. The X and home affordances inside use-case demos sent users to unexpected external pages.
Misunderstanding of the product
The marketing page never made clear what 'Paks' are or what is bundled with Cloud Pak for Business Automation, leaving users guessing about scope and pricing.
Misinformed signup
A 48-hour delay before trial access killed momentum. Users expected immediate access while their interest was at its peak.
Misinterpretation of content
Benefits and capabilities lacked depth. Users wanted clickable benefits, embedded studies, and clearer terminology around 'demo' versus 'trial'.
Overwhelming content
Pages were heavy with text and required excessive scrolling. Static imagery felt unengaging — users asked for video and motion to bring use cases to life.
05 // Findings
What we saw, in depth
The Walk Me tour didn't carry users far enough
Upon logging in, users were dropped into a walk-through but still felt unsure what to do next. They wanted a guided, use-case-driven onboarding — not a generic tour of the UI chrome.
- ›Users wanted a 5-minute tutorial that ends with them building their first automated workflow.
- ›Getting Started simply replayed the Walk Me tour, adding to the confusion.
- ›No on-demand technical support or chat was available from the homepage.
Replace the generic tour with a condensed, use-case-driven walkthrough plus a progress checklist and on-demand chat support.
Trial navigation kept dropping users out of the product
Users were uncertain how to return to the trial homepage from inside use-case demos. The X and home buttons inside demos redirected users to external sites instead of the trial.
- ›Users assumed home / X inside a demo would return them to the trial dashboard.
- ›Instead they were sent to an unrelated page, breaking the trial flow entirely.
Anchor every demo inside a predictable trial shell with a clear, persistent path back to the trial homepage and the task checklist.
Users couldn't tell what was actually inside the Pak
The marketing page never clarified which products, integrations, or services come with Cloud Pak for Business Automation, leaving users to guess at scope, pricing, and value.
- ›"Is it a package of a product or one product?" — Marketing Operations Manager
- ›"What tools does this integrate with — Netsuite, Salesforce — or does it replace them?" — System Admin
- ›Pricing existed as a tab but never surfaced a real number or comparison.
Add a visual side-by-side of bundled products and a clear pricing table so users can quickly compare and decide.
A 48-hour wait killed the trial momentum
After completing registration, users were forced to wait 48 hours before accessing the trial. By then the spark that drove them to sign up was gone.
- ›Users called out the delay as a deal-breaker, citing competing tools that grant instant access.
- ›Trial signup copy felt dated, text-heavy, and visually flat.
- ›Users asked for hyperlinked use cases and a bolded 30-day trial callout.
Grant instant trial access and redesign the signup with more visual hierarchy, less copy, and clearer trial benefits.
Benefits felt vague and clients studies felt static
Users wanted to click into benefits, see proof, and watch use cases in motion — not scroll past static blocks of text.
- ›"It'd be nice if I could click a benefit and see specifics in action." — Marketing Operations Manager
- ›"Make it more interactive. Separate Benefits into its own page." — Business Analyst
- ›Users couldn't easily tell where the demo lived — Explore the Trial or Speak to a Consultant?
Combine Benefits, Capabilities, and Services into a single interactive surface with embedded video and clear navigation to the demo.
Marketing pages were a wall of text
Users were fatigued by the volume of copy and the scrolling required to find demos and videos. Only one user successfully located the demo content tucked under Get Started.
- ›"The service is engaging — the content is not." — Business Analyst
- ›Static imagery underperformed; users repeatedly asked for video or GIFs.
Replace dense text blocks with motion, condense the page, and surface demos and videos at the top of the experience.
06 // Framework
Nielsen's 10 usability heuristics
| Heuristic | Definition |
|---|---|
| Visibility of system status | The system keeps users informed of what is happening through clear, timely feedback. |
| Recognition rather than recall | Minimize memory load so users don't have to remember information across screens. |
| Help & documentation | Help is easy to find, approachable, and doesn't pull users out of their flow. |
| Error prevention | Actions in the system should be clear before users commit to them. |
| Recognize, diagnose & recover from errors | Errors present clear recovery paths back to the user's previous state. |
| Consistency & standards | Use consistent language and interaction patterns across the platform. |
| Aesthetic & minimalist design | Focus the interface on essential actions and information. |
| User control & freedom | Support undo and redo so users can recover from mistakes. |
| Match between system & real world | Speak the user's language and follow real-world conventions. |
| Flexibility & efficiency of use | Simple for novices, fast for experts. |
07 // SUS
System Usability Score
After calculations, participants rated the trial experience with a SUS Score of 46.5 — a clear signal that the current trial flow falls well below acceptable usability and requires structural change, not surface polish.
08 // Conclusion
Where we landed
Users were excited by the product but blocked by the path to try it.
Onboarding needs a use-case-driven walkthrough — not a tour of the UI.
Marketing copy needs to be condensed, visual, and explicit about what is bundled.
Trial access should be instant; a 48-hour wait erodes intent.
Benefits and use cases need to be interactive and embedded, not static.
Stakeholders are fully briefed on the findings — limit content, surface valuable use cases, and rebuild the SaaS trial experience. Design sprints are organized around each theme to ship an exceptional trial experience for both new and existing users.
IBM ▪ Cloud Pak for Business Automation