Challenge:

‍

Improve retention and conversions by understanding the user journey and addressing customer complaints

A UK-based AI-powered finance app that helps users manage their personal finances saw significant growth in 2023, expanding from 100 to 250 employees, securing a major funding round, and reaching over 800,000 monthly users.

The UX Researcher at the company, passionate about customer feedback data, has dedicated time to monitoring the user journey, analyzing product and retention metrics, and using insights to guide the company’s roadmap and enhance user experience.

"The company is highly data-driven, closely monitoring metrics like LTV, conversions, app store reviews, CSAT, and NPS at various touchpoints. When these metrics decline, it impacts our strategic business goals. It’s crucial for us to understand the full user journey to help the Product and Engineering teams identify and address key pain points," explains the UX Researcher.

As he analyzed the user journey, he realized that traditional product analytics platforms didn't provide a comprehensive understanding of the user experience. 

"We've always used product analytics tools, but they only gave us quantitative data. To fully understand the user journey, we needed qualitative feedback as well,” he says. 

While customer interviews offered qualitative insights, they were unscalable and time-consuming.

"We conducted many interviews to identify potential issues but not to quantify them. Even with 15-minute interviews, building a large enough sample size to determine if an issue is significant would take too much time. This approach has a high opportunity cost, consuming a lot of time that could be spent more effectively."

‍

‍

Realizing that their data wasn’t being used to its full potential, the team partnered with Chattermill to analyze user feedback at scale and maximize the opportunities for teams to understand the user experience across the entire journey and resolve issues impacting key business metrics.

‍

Solution:

‍

Customer feedback analytics designed for Product and UX teams

Chattermill unified the company’s feedback from NPS, CSAT, app reviews, and cancellation surveys. They could then turn this into insightful reports and dashboards used across the company to scale CX across product development. 

Identify major pain points and drive stakeholder buy-in

Over 80 internal users across Product, UX and Engineering teams use Chattermill to pinpoint product quality issues that need fixing.

"Chattermill helps us identify key issues across the product, prioritize and rank the most significant problems, and track feedback volume and sentiment changes over time. This enables us to monitor user experience, understand negative sentiment drivers, and proactively investigate potential issues before they escalate,” explains the Lead UX Researcher. 

“Additionally, quantifying these issues helps us prioritize roadmap initiatives and convince the business that these projects are worth pursuing."

‍

Validate hypothesis and A/B tests

The teams also use Chattermill to experiment, test, and validate hypotheses.

"We might roll out a feature to 40 users and then analyze their responses in Chattermill compared to a larger segment,” explains the Lead UX Researcher. 

“We track whether it results in fewer negative comments or more positive feedback. We also monitor whether it leads to increased LTV or improved retention. Chattermill is a key tool in that type of analysis."

‍

Optimize digital user journey

Product, UX and Research Teams use Chattermill to continuously monitor and enhance the digital user journey.

"We identify key user journeys and if we see significant conversion drops on certain pages, we highlight user complaints related to those areas. This helps us prioritize fixes and enhance the user experience, which is as important as launching new features. It's a valuable tool for pinpointing where to focus our efforts."

‍

Inform the product roadmap

Finally, Chattermill is essential for gathering product insights for their product roadmap.

"During OKR planning, team members seek insights from me, focusing on metrics related to financial performance or app usage. When brainstorming initiatives, they ask, 'What are users saying about this feature?' I then filter through the data in Chattermill to extract specific comments on relevant themes to share with them, which helps shape the scope of the roadmap," explains the Lead UX Researcher. 

In addition, they find the platform easy to use, which enables everyone to make data-driven decisions at a rapid pace.

‍

‍

“Users can easily navigate charts to identify major issues and access AI-generated summaries of user feedback by clicking on specific items. This allows anyone to quickly get up to speed on the challenges in that product area, regardless of their technical expertise."

‍

Results:

‍

50% reduction in customer complaints

With the ability to monitor the user journey at every touchpoint, te company could uncover critical issues affecting their retention metrics.

‍

‍

"One solution was adjusting our express fee - charged for immediate money disbursement - to allow larger cash advances while managing repayment risks."

Ultimately, the team made significant improvements to address customer pain points. But, they still had to remain aware of the potential impact of changes on customer sentiment. 

"We increased cash advances significantly, alongside raising the fees. Additionally, we raised the minimum cash advance by 100%." explains the Lead UX Researcher. 

“To monitor feedback on both fees and cash advance amounts, we relied on Chattermill."

"We were relieved to see a 50% decrease in complaints about cash advance amounts, while feedback on express fees remained steady. This insight revealed that users cared more about advance limits than fees. With this reassurance, we proceeded confidently, knowing we could improve our product without concerns about negative reactions to the fees."

Now that the team could trust their feedback data, they could be confident that it would drive up retention metrics.

"Once we increased advance amounts and noticed fewer negative comments, we inferred that this would eventually lead to better retention - and it did. Using Chattermill, we can act more quickly rather than wait three months to see if our retention improves. It's a useful way to accelerate our decision-making and respond to user feedback more effectively."

With standardized analytics and reporting in place, the company is now seeing the full potential of its feedback data. 

"You can operate without feedback analytics for a long time, but then you can easily miss these important issues. You might think you're doing fine, but without understanding your customers' needs, you risk falling behind,” says the Lead UX Researcher. 

“Over time, this can lead to stagnation and a lack of innovation, making it difficult to know what to improve. In the long run, that can put your business in a tough spot."

Integrating Chattermill into their UX research proved to be a strategic move for the business, resulting in enhanced business metrics and efficiency.

‍

‍

Discover how Chattermill can help you leverage customer feedback to optimize the digital user journey – book a demo today.

‍