How might we increase the number of accounts that are actively using Policy?

Our hypothesis was that it is difficult for customers to get onboarded to Policy because they don't understand its full capabilities. To get onboarded to the product, customers need the help of Sales or Customers Success Managers. From January to July of 2019, my PO investigated churning customers. There were 544 customers churning. Of these customers only 10% had 10 or more policies setup and 67% had 0 policies.

Why are users not using Policy?

• Since customers need to reach a certain maturity, there is very low awareness of Policies functionality from prospects, accounts on trial or accounts with no subscriptions.
  
• Policy can be intimidating due to the amount of technical knowledge that’s needed.
• Policy creation and management is not scalable enough for enterprise customers.
• It is not easy to monitor the current status and progress over time.
• The lists of active policies are growing big and overwhelming.
• Policy does not adapt well to different workflows. (e.g. users have to manually share policies).

What is the visitor behaviour in Policy?

• My policies is the most visited page. This page is where users are managing all created policies.
• Almost all of Policy visitors are Siteimprove Super Users (8 or more days active).
• Policy overview page, which is a dashboard is not used that much because users don't get much value from the data shown on the page.
  
• The policy library is a page that provides templates for customers to get started with Policy. But, we saw that only 17% of the visitors added a policy from the library. 83% left Policy Library page without adding a policy from the library.

A product roadmap was developed based on the user research. Each issue that we chose to address was broken into user stories in our backlog. My PO and I decided to address issues with the My Policies page because users are actively using this page the most.

Although, this page is used the most by users we identified some issues with My Policies page. For users, it is not easy to monitor the current status and progress because there is no indication of what changed overtime. The list of active policies are growing big and overwhelming. A lot of the policies get outdated overtime, or some policies may get put on hold and later brought back into focus but there is no way to archive a policy. Also, the page has some known Accessibility issues and is not aligned with the new design system.

Based on the insights from the discovery phase I wanted to introduce some new features to help users proactively manage their policies.

• Deactivating and reactivating policies for users to be able to manage their overwhelming list of policies.
  
• Adding change data based on the last 30 days for users to visualize what has changed.
• A tab for users to manage their favorite policies instead of favorite policies being pinned to the top of the list.
• An accordion functionality for users to easily access notes and details about a policy.
• Potentially introducing a new filter system to accommodate the growing list of filters. This was an initial test on introducing a new filter system.
• Changing from a table view to a list view to have more focused data and visualizations.
• Being able to sort by policies with matches that have the highest or lowest increase.

I involved the product team to get feedback on the designs as well as set up design critiques with other designers. After doing 3 concepts, the team decided on concepts to test with users.

The UX writers helped ensure all copy aligned with user intent and was intuitive to use. I recruited users with the help of my PO and Customer Success. I wrote a script, created a prototype in InVision and conducted remote usability testing sessions with users. The usability session revealed that users liked how visual the page is now and being able to track the progress over a given time. Users liked the functionality of being able to deactivate a policy and view details of a specific policy within the page. Users struggled with using the new filtering system and thought the page had more scrolling.

I made updates to the designs after user testing and additional internal refinement sessions.

• I condensed the list view, so it is easier for users to scan all their policies. I aligned the list view to the guidelines written in our design system.
  
• Instead of an accordion I chose to use a side panel to be able to present quick actionable data to customers about their policies.
• I kept the existing filter system since the design system team is working on creating a new filter behavior to be aligned across products.
• I changed the naming of the archive functionality to make it clear that policies that get archived would be made inactive.

Since we released the new my policies page, the usage of that page and adoption of Policy have increased significantly by 150%.