Synthesizing qualitative sources to with a structured AI reporting suite. Collapsing tedious research into a click of a button.
Role
Lead Product Designer
Time Frame
2 months
TL;DR
In 2024, CB Insights wanted to position itself as a leader in AI for private companies. An initial attempt at an AI assistant had failed. It had a poor usability, and unclear value. I conducted user interviews and usability testing to diagnose confirm which areas needed improvement. Then took those learnings to redesign and relaunch the app in the span of 2 weeks.
In the process of usability testing, I surfaced critical usability gaps, reframed the project around one clear user value (AI-generated research reports), and drove alignment across teams. The result was Instant Insights: a one-click reporting feature embedded directly in company profiles.
Impact:
Grew 4th most used feature on the platform shortly after launch
50%+ retention
Key driver of client renewals
Background
Generative AI has been a focus at CB Insights since the launch of ChatGPT in 2022. As a research company that covers the business around AI, it was important to be a thought leader in the space and show the successful adoption of generative AI in the company. The embodiment of AI? Chat. So, a solution was rushed into development based on an externally prescribed solution.
Before even launching to customers, it became clear that the solution was convoluted and hard to use. So our team moved forward to collect user feedback and turn the project around.
My Role
The original project idea that I came up with and pitched.
Led all design for the project.
Led usability testing and supported other user interviews.
Reframed the solution, pushed to scrap the chat metaphor and focus on AI reports embedded directly in company profiles. Crucial to the success of the product.
Managed a front end developer in implementation of the designs.
The Core Problem
Back in February 2023, I had put forward the project for a company-wide generative AI Hackday. The problem statement was simple:
Our core datasets, deals & business relationships, were being used to a fraction of their potential. By not utilizing our underlying news coverage for the datasets, we were having our clients put in hours of extra research time to get the full value of our data.
The high-level solution I proposed was to have genAI analyze the news coverage and report out any important information. Essentially short short-circuiting any client research time and delivering analysis instantly. It was an easy application of AI, yet it would have immediate value to our users. The solution won Hackday, but was not implemented as is.
The Problematic UI
When going into initial development, the solution had been re-imagined as an AI product suite. It would be an assistant that would sit on top of the platform to help you complete tasks. It would have a series of features, from company list building to drafting emails to the aforementioned AI reporting. Being an AI assistant, the interface had to look like a chat, even if none of the features were chat-driven.
The result was a convoluted solution. Multiple product and engineering teams were involved, driving in different directions. It became clear before launch that it was not salvageable.
The Turnaround
In the middle of this project, a new CEO was brought into the company. In order to avoid a failed launch, my team was given free rein to do whatever was necessary to make a successful feature. The catch was that we still had to make the original launch date.
To turn the project around, we needed feedback from our end users, so we suggested launching the feature to a limited set of users to test it out. Simultaneously, we would also run usability tests to assess specific pain points in the UI.
We recruited 30 client accounts (roughly 70 users) with the help of our account team, who we would all be monitoring throughout. I then worked to separate five of those users to conduct in-depth usability tests with.
Usability Testing
Throughout the testing process, I documented user characteristics as well as reporting out on usability feedback.
After running through the usability testing, the results were clear:
The content is great
Despite all the issues the users ran into, they loved the outputs from the AI reporting. Eliciting quotes like:
"
Yeah, you just saved three hours on Monday
- Deal Maker client after reading a Scouting Report.
Another user was looking through the reporting and explaining how it matched up completely to what he needed for his reporting.
The chat metaphor wasn’t working
Users are expected to be able to chat. Many tried typing or clicking the welcome message repeatedly.
They were frustrated when they realized it was limited to pre-defined options.
Navigation confusion
Navigation was independent of the platform's information infrastructure and introduced its own secondary navigation that confused users.
The navigation was inconsistent: sometimes controlling the platform, sometimes reacting to it, sometimes working independently. Because of that, the users struggled to build an understanding of how to interact with the solution.
Poor readability
The scouting report in particular was long (7-8 page reports) but confined to a 320px width sidebar.
Scrolling was tedious, making valuable insights hard to consume.
We had a clear insight: “The content is great. The interface is not.”
I presented the findings and proposed a solution to the team and the VP of Product: We needed to get rid of the chat UI completely and focus on the AI reporting. Frustration had already built up internally on the existing solution, so we walked out of the conversation with complete alignment.
Reframe The Solution
At that point, we had very limited time and resources to get to a launch. This meant we were limited in what we could introduce. Any redesign of existing pages was off the table.
In addition, I had three obvious points that informed my design thinking:
All of the reports in the sidebar had a natural place in our company profiles.
Scouting report corresponded to a full company report, and belonged at the top level of the company profiles.
The deal and business relationships report belonged to the deal and business relationships sections of the profile.
The initial solution had focused heavily on enriching our advanced search. This was a space that was only used by a handful of power users, so we needed to deprioritize it.
Everything outside of AI reporting was not obviously valuable and should be put on hold.
So I came up with three potential ways forward:
Embedding the content into the existing company profiles
This solution came with too many issues. Our data tables were not built for long text content, and adding the content would make other data comparisons difficult. The Scouting Reports were 8-page reports and had no natural place on the profile.
This would be the ideal solution if we could redesign the company profile, but we simply didn’t have the resources or organizational will.
A reader on top of the company profile to access the content
This solution would require some level of secondary navigation instead of relying on established IA, so we were wary of going this direction.
[ Embed CTA buttons in the appropriate sections of the profile that unlocked the AI reporting]
This was the ideal combination of using the platform IA and being feasible from a development perspective.
Going with the embedded CTA buttons, we would redesign the sidebar that was already in place and remove any secondary navigation. We would then build the buttons into the company profiles, essentially just requiring us to insert a banner into existing sections.
After having built conviction in our solution, we presented the solution along with the above lo-fi designs to our leadership board and were able to get immediate buy-in.
Collaborating with our CEO, we the feature renamed Instant Insights.
Design Decisions
Now that we had leadership behind us, we had to dial in the designs and build. There were 7 days left until we would release to clients.
Dark mode
CB Insights didn’t have any dark mode designs, so we needed to set guidelines around the implementation. Because of the short timeline, we relied on Google Material and our personal tastes. That allowed me to rapidly improve on the look and feel and set a future precedent for dark mode on the platform.
We avoided going full dark mode and maintaining white cards. This had a lot of debate, but ultimately going full dark mode didn’t match the platform as it was at the time.
The "Magic" Button
The requirements for the magic buttons were simple. It needed to:
Be highly visible on the company profile page.
Have concise copy and a clear call to action.
Indicate clearly that this was an AI report.
We tried every color combination available in our color palette, but ultimately we landed on the dark look since it matched the sidebar design. It stood out on the page, but I didn’t match the rest of the page very well. However, we needed to lock in quickly, so we committed.
We now placed the buttons alongside the content they belonged to. That meant there would be six buttons for each profile (company report, fundings, investments, exits, acquisitions, and relationships). Only copy and their placements would differentiate them, but the content in the sidebar would depend on the section it was in. We were worried that users wouldn’t be able to tell the difference between the buttons, but it turned out that they picked up on it very quickly.
To communicate that these were AI-generated, we decided that a sparkle icon, copy, and some disclaimers in the sidebar would be sufficient.
Key themes
In addition to all the other changes, we also added a new content type. We already had insights into each relationship/deal for a company. Now we would also create an overall summary telling the user about patterns in a company’s relationships/deals.
Since each data type had different strategies and outcomes we would provide key themes separately for fundings, investments, exits, acquisitions, and relationships.
It slotted in very naturally into the design. It would act as a summary leading into the list of individual relationships.
Complementing with context and data
To improve the contents and navigation, we added data and context where it would help.
All AI content was labeled with a generation date since it would change as new data came in.
Added logos to reports for visual flair.
For deals and relationships, we provided the contextual data to help the user scan the relationships.
For scouting reports, we provided some general data about the company to give context to the company and the report. Since genAI isn’t great at reciting data we felt this was more reliable approach to key metrics.
Sourcing & Verification
Citations were critical for clients to trust our AI reports. Every fact needs to be verifiable, and just the presence of citations has helped solidify it. The deal and business relationships insights were always tightly coupled with their sources. Allowing the user to dig investigate the underlying data whenever they needed. Since Scouting Reports were built differently we had to build out a separate feature and approach for sourcing them. However, we knew it was crucial so we pushed to push the feature out a week later.
Streaming
Streaming was particularly an issue for Scouting reports. Since they were very long and referenced many of our datasets it could take over a minute for us to stream the contents. Instead of waiting for the entire report to complete, we would stream the text like an LLM. This allowed the user to start reading the report within seconds, even if it took a minute for the report to load completely.
That also meant that we needed to communicate clearly when a report was getting ready to stream, when it was still generating, and when it was complete. We didn’t want users to copy partially generated reports. So we had to disable all export functionality while the reports were still streaming.
Caching
It would be wasteful for Scouting Reports and Key Themes to be generated fresh every time the user clicked the magic button. It would force the user to wait and also rack up costs over time. We knew that our top companies were fairly concentrated, so it would be safe to assume users would request many of the same reports. Because of that we chose to cache the reports. After one user generated a report, the next user would load the full report in seconds. To keep the reports fresh, we would require a new report to be generated if anything changed on the company profile.
Final Product
Results
The reactions and feedback to Instant Insights has been overwhelmingly positive. Nine months later we still hear clients praising the reports. At the end of the year, the commercial team has repeatedly cited the features as one of the key reasons our clients renewed their contracts with CB Insights.
Instant Insights became the 4th most used feature on CB Insights after a tiered rollout (hence the usage curve below).
Retention held above 50%, which was unexpected since we thought this feature would slot into the early part of the client’s buy/investment/partner process. It turned we’re helping their entire process.
Cited by the commercial team as a key factor in client renewals.
Instant Insights transformed a near-failure into a flagship AI product. Ultimately the feature was launched too late to position the company as a leader in generative AI, but we ended up launching a higher quality product than any of our competitors.
The most gratifying feedback came when a prominent investor, and one of our biggest critics in usability testing and the early access program, reached unprompted to our the CEO and our founder to thank him for the new features.
Reflection
This project reinforced a few lessons for me:
Introducing a temporary pattern is a slippery slope. The magic buttons were intended to a be a temporary solution give the time we had to build. The intention was always to find a natural place for the content and merge it into the platform. However, after the success of the launch there was complete aversion to changing the pattern. In fact it would continue to be perpetuated by other teams in other areas of the platform.
If you have a working Information Infrastructure; Use it. This one might seem obvious in retrospect, but the failure of the initial analyst fell almost fully on the fact that it tried to create a new system of navigation and IA.