In 2019, the Digital Transformation Office piloted three different methods to measure online task success:

  • usability testing
  • task funnel analysis
  • task completion survey

Measuring online task completion helps us understand how design improvements affect task success. It also helps institutions adopt a top task approach to managing online information and services.

“Change the key metrics and you will change the organization. Change the organization and you will deliver vastly better digital services.”

The pros and cons of each method

Here’s what we learned about each method.

Usability testing

Moderated usability testing is a fantastic way to learn about how people use your site, and to see where there are problems. It’s also the gold standard for measuring task success. But, usability tests focused on task performance are a bit different from standard usability testing. The main difference is that task performance testing tends to use more participants (at least 16). It focuses both on arriving at a reliable success score per task and on learning about how people use the site.

Moderated usability testing results are difficult to compare across many institutions. They rely on phrasing tasks and scenarios consistently. As well, the cost, time and expertise needed to run and analyze rigorous usability tests can be challenging for smaller institutions.

That said, nothing can replace watching real people use your site. We always recommend incorporating usability testing into your web team’s toolkit.

Task funnel

Task funnels use anonymized web analytics to track visitors’ paths from beginning a task, to the successful completion.

Funnels work best for tracking conversions over multiple pages where there is a single sequence of steps to follow (such as an e-commerce checkout). The drawback is that people drop out of funnels for reasons that have nothing to do with the effectiveness of the experience.

This method is not great for answer-finding tasks. These often have multiple path options making them difficult or impossible to track.

This method can be challenging if analytics platforms aren’t robust enough or you’re unable to track users through secure areas of a website.

Online survey

The survey method is the easiest to implement and rerun year after year. In fact, the Digital Transformation Office developed a survey that any institution could use to collect task success data. It works by asking visitors what they came to the site to do, and whether they were successful.

Results from these surveys can be automatically transformed into dashboards. This further reduces an institution’s workload for compiling and analyzing the data.

The Digital Transformation Office survey prompts visitors to:

  • select the task that they came to do from a set of options (including a write-in option in case their task wasn’t listed)
  • confirm if they were able to complete their task
  • select why they were not able to complete their task if they couldn’t (including a write-in option)

We found that this method allowed the best balance between:

  • standardizing data
  • understanding why people visit certain pages
  • pinpointing the parts of the online experience causing service delivery issues

Simplifying methods for better results

Experimenting with multiple methods to measure top tasks highlighted how different methods might create inconsistent data. It’s hard to compare the results of a task funnel to a usability test!

The online survey method provides a standardized data collection method. It is technically easy to implement, and provides clear data about top task strengths and weaknesses.

The feedback we heard from institutions using this method was positive. It indicated a culture shift to top task management.

“The survey allows us to prioritize which section on the website should be optimized first. For example, our focus/energy should go towards top tasks that are not performing as [well] as the other ones.”

“Over the course of time, the findings of the survey have played a part in the evolution of the website.”

We want to hear from you

What would make online top task reporting easier for you to complete?

  • tips and best practices for writing top task survey questions
  • advice on how to break down big tasks into smaller, measurable steps
  • instructions on what to do if your service includes online and offline delivery channels
  • other suggestions?

Inspired by what you learned? Share this post with your team.

Connect with the Digital Transformation Office at TBS:

Learn more