18F/18f.gsa.gov

---
title: "Areas for growth part 2: mixed methods"
date: 2020-02-27
authors:
- victor-udoewa
tags:
- user research
- how we work
- lessons learned
excerpt: "Mixed methods is an approach in the social sciences in which you gather
both quantitative and qualitative data in an effort to make more
informed and integrated interpretations based on the combined strengths
of both types of data."
---

*Usually organizational blogs, especially blogs by civic tech
organizations, focus primarily on sharing successes. However, we have
found that we actually learn far more from the failures of others than
from their successes.*

*So, we created this series to share our areas for growth, in hopes that
you can learn just as much from our process as from our successes. This
series was developed from ongoing conversations by a diverse group of
18F members, including both term-limited and career employees who have
varying levels of experience and tenure (e.g., freshly arrived to more
than 5 years). We joined together to talk about ways to improve our
techniques, approach, dynamics, and strategy. Part 2 of our series is
about mixed methods.*

It is natural for design researchers to use qualitative methodologies in
their work. It is what they are trained to do as they attempt to answer
questions about needs, values, emotions, desires, problems, and
complexities. It is also natural for a data scientist or statistician to
use quantitative methodologies to answer questions about statistical
trends, how often an event occurs, or how many people experience a
phenomenon. We have learned, however, that there is power in bringing
the two methodologies together for a mixed methods approach.

Mixed methods is an approach in the social sciences in which you gather
both quantitative and qualitative data in an effort to make more
informed and integrated interpretations based on the combined strengths
of both types of data. This approach allows a researcher to triangulate
information to create a rich, more complete narrative of an experience
or situation. Beyond understanding a person’s experience with a service
or product, mixed methods also help to identify repeated patterns and to
what extent this pattern is demonstrated among the people using a
service or product. Lastly, this approach provides a clear way to
connect the value of a service or product to the human users (which is
often qualitative and sometimes quantitative) to the value it has to an
organization (often quantitative).

When surveying our projects, it’s clear that we could use a mixed
methods approach more often. There are definitely projects that may not
have needed it. There are also projects that could have used it. By
adding quantitative research to the qualitative research that was
already done, the project’s interpretation and recommended strategy may
have been further validated. However, it also may have changed, colored,
or tweaked the findings in such a way that the final recommendations
took a slightly or largely different direction. Using mixed methods
allows us to gather information at varying scales, depths, and types in
order to form a holistic picture to better inform our design choices.

## Three Approaches

There are three high-level general approaches to employing mixed
methods, which are: a convergent approach, an explanatory approach, and
an exploratory approach. The level of intention varies between the
options.

In a convergent approach, we create research goals and then research
questions. From the beginning, we intentionally plan both qualitative
research and quantitative research based on our questions. We look for
the similarities and differences between the quantitative and
qualitative data, and use the insights to generate a report showing our
findings and recommendations.

In an explanatory approach, we also create research goals and research
questions. However, here we plan a quantitative study based on our
questions in order to look for patterns among the quantitative data of
relatively large numbers of people. After studying this, we may have new
questions about why something is happening or about the emotions or
needs of the people experiencing a particular event. To answer these
questions and *explain* what we are seeing, we design a qualitative
study to complement our quantitative work. When we report our findings,
the quantitative data is enriched by the explanatory qualitative
insights.

In an exploratory approach, our research questions lead us to plan a
qualitative study to consider the experiences and stories of people who
use or could use a service or product. Through synthesis of qualitative
research, we develop hypotheses that we can *explore* by planning an
experimental quantitative study, usually with a larger group. We then
report on our qualitative findings and quantitative results.

## Call for Help

It’s not necessary that every design researcher be able to conduct both
qualitative and quantitative research. What is helpful is every design
researcher being able to recognize the need for quantitative research or
a mixed methods approach, and to know when to invite the help of a data
scientist. Using a mixed methods approach and working with a data
scientist on quantitative analysis can ultimately increase the
research’s validity and uncover insights that provide a more complete
view of what is happening. In the end, the public who use our services
and products will benefit.