In May 2023, I sat down with Jackson Ridl, Product Manager at Microsoft, to discuss his work in creating the Microsoft 365 suite of free productivity tools, an endeavor that opened access to crucial tools whilst creating new opportunities for the company. To ensure that pursuing a Freemium model alongside their existing paid membership models was a smart business decision, Jackson and his team relied heavily on experimentation, data science, and creative socialization of insights. 

In our work with Cultivators over the years, we have noticed that many employees with incredible ideas for communities, products, and culture initiatives are unsure of how to collect and communicate data to show impact and win buy-in. 

You can read Jackson’s full story here, but while we had him on the line, we wanted to deep dive into how to use data science to drive your change initiative forward and share some resources!  

Know the data that matters to your company

In Jackson’s case, he and his team needed to prove that a Freemium model wouldn’t cannibalize their existing paid business. So, they looked at the impact of the free products on things like payment flow click-through rates and how likely new users were to upgrade from a free to a paid subscription. Look at your company’s central metrics or goals and think about how your idea or initiative could impact these metrics. Anything from revenue to engagement growth to employee retention can support your work. Aligning with key metrics will make stakeholders more likely to pay attention and lend support.

Reflection Question: Pull up your company’s OKRs, KPIs, or major goals. What is your hypothesis for how your idea impacts one or more of these metrics?

Collect (and use!) a blend of Qualitative and Quantitative data

Both are important! Jackson notes that “Sometimes product managers can get lost in the metrics and be so focused on the data that they forget to zoom out and look at the qualitative user research to understand the heart and the soul of their product.” Here’s how you can leverage both:

Qualitative data

Jackson and his team used Usertesting.com to gain familiarity with the customers they were seeking to serve. “We set up some wireframes of how we wanted to position our purchase journey to see if customers would react positively.” Testing this with potential customers helped them improve their purchase journey in ways that resonated most with their users. Qualitative data allows for perspective and context beyond the numbers and can be a crucial part of telling the story of your idea’s impact.

Reflection Question: Who is the ideal “customer” for your initiative? Remember, if you’re running an internal initiative, your customers may be your colleagues! How might you get their initial feedback on your idea? What are the big categorical questions you want to ask them? What are the smaller questions that can lead to insights and context about those big questions?

Quantitative data

For his work, Jackson and his team looked at both local data (ex: How many people are clicking this “buy” button? How many people are scrolling down to the bottom of this experience to see what it looks like?) and global data (ex: How many people bought Microsoft 365 in total?). This allowed the team to understand how their experiment was impacting the overall product, and to determine if the effect was statistically significant. This allowed them to clearly show their impact on the overall business and connect it to local data events when presenting their data to the organization.

Reflection Question: Look back at your answer to the first question. What data could you collect? How will you collect it? Are there multiple vantage points to view the data with?

Use tools that support your work

In a data-driven company like Microsoft, it wasn’t enough to collect data, Jackson and his team also needed to ensure that they could prove their initiative had a statistically significant impact. To do this, they used open-source statistics tools like Optimizely to determine the sample size they needed to reach statistically significant results and determine how long their experiments should run. Need some support in getting started? This article is a helpful overview of how to use statistical analysis in a corporate setting.

Reflection Question: What tools are open to you for data analysis at your company? Are there free tools that could support your work? Do you have a colleague who is a data whiz who could be your co-conspirator? Could you ask them for support?

Use data to build your confidence

Because of their diligence in collecting data throughout their work, Jackson and his team felt confident in the impact that their initiative was having. Jackson says the goal is to be able to say, “The data moved, I understand why, it's positive for the business, and the customers like it.” Picture yourself walking into your next stakeholder meeting feeling certain of the positive impact you’re making! You’ve got this! 

Collecting data that proves your impact is the single most crucial aspect of moving your idea for change forward. Data may seem scary or insurmountable at first, but we hope that these tips empower you to harness the power of data to drive your company forward. And if you need some help creating a compelling data plan or telling a compelling data story, head to our Cultivators Community for support, guidance, and resources.

Note: Any opinions shared by Jackson are his own and do not represent the views of Microsoft. Any learnings or information presented are his own based on his experience at Microsoft.