Workplaces, hospitals, schools, and other institutions are rethinking the way they conduct their regular activities in response to the COVID-19 pandemic. As government agencies continue work on digital service projects, they also need to rethink how to best gather the needs of people and incorporate them into product development. Instead of hitting pause on user research plans, now is the time to understand the benefits of remote research.
As a remote-first company, Ad Hoc routinely conducts user research remotely. User research helps us ensure that we’re meeting the real needs of our users and also reduces redundant work as a result of incorrect or flawed assumptions.
Advantages of remote research
The good news is that remote research offers many advantages even when we aren’t facing a pandemic. Here are some ways remote research helps Ad Hoc improve the products we design for our government customers:
Faster insights at lower cost
If you eliminate travel time, researchers can deliver insights faster. All of our work follows agile software development methodologies, which depend on rapid cycles of learning. With remote research, travel is never a bottleneck to getting results or a time sink taking away from project work. It also has the added benefit of saving our customers, the federal government, the expense of travel.
Better data
When you conduct research remotely, researchers are able to catch users in their natural environments — at home or in the workplace. Performing research in an environment where participants feel most comfortable is often the key to meaningful insights. People can reference the sticky notes they keep in their workspace, and we can get a glimpse of their typical workday, including common distractions like colleagues coming into their office during a session. Just being in a familiar space puts users closer to the mindset we’re trying to observe. Remote research can also be less intrusive. Participants can be more at ease when there is no camera on a tripod, audio recorder, or person next to them with a clipboard. This means more realistic observations, and more realistic observations mean better, more accurate results.
Easier for people to observe
At Ad Hoc, we like to approach user research as a team sport; we encourage stakeholders and other partners to observe research sessions as much as possible. Conducting research remotely makes this easy. There is less potential to overwhelm participants with unfamiliar observers since they’re almost invisible in a remote web conference session.
Access to a wider range of participants
Access to a diverse range of participants is a key component to meaningful user research. When conducting remote user research, distance and geography are no longer barriers. As long as participants have an internet connection and a computer, remote user researchers can reach our users wherever they may be located. This can be key in eliminating research bias compared to traveling to a region and doing a handful of in-person sessions that are clustered together.
Less participant burden
Preventing participant fatigue and keeping users engaged is a major concern for all user researchers, both in-person and remote. Rather than asking participants to spend time traveling or letting unfamiliar researchers into their home or workplace, remote research mitigates a lot of these factors by simply asking participants to connect to a remote session or follow a link to an asynchronous activity from wherever they are.
Tools for remote research
Even when we aren’t being asked to practice social distancing, there are a multitude of reasons why remote user research can be a good idea for research teams. One last important topic to discuss, however, is how to tackle the technology related to performing remote research. Finding and mastering tools that support and enhance user research is something all researchers are familiar with, but performing research remotely can exacerbate the need for good tools to perform your craft. Below, we have included several lists of tools that we’ve found useful for remote user research.
Participant management
In order to keep track of our pool of research participants, we’ve used the following:
- Ripple — Web application intended for managing a research panel
- Airtable — Flexible way to create relational databases that can be used for just about anything
Virtual phone
We set up a shared phone number to use when we’re contacting participants by phone and text. This allows us to give participants a single point of contact and allows multiple remote team members to see the whole chain of communication when recruiting for a study.
- Grasshopper — Virtual business phone system, allowing multiple users to access the same phone number
Scheduling apps
These tools take some of the manual work out of coordinating schedules with participants.
- Doodle — Flexible web-based polls that can be used to ask participants to indicate multiple times when they could be available
- Calendly — Connects to the organizer’s online calendar and shows available times for participants to pick an appointment
Virtual meeting apps
Remote user interviews and usability sessions are done via virtual meeting/screen sharing applications. Depending on the contract, we’ve used the following for recording and screen-sharing during sessions.
- GoToMeeting — Can automatically process transcripts from recordings and produce an interactive transcript that syncs with the screen video
- JoinMe — Simple and allows participants to control the researcher’s keyboard and mouse from Chrome without requiring any software installation
- Zoom — Easily handles large numbers of participants and provides advanced functionality like in-session polling and breakout rooms
Prototyping tools
In collaboration with our design teams, we create and test prototypes using these tools.
- Sketch — Detailed screen design tool
- InVision — Web-based prototyping
- Axure — Highly interactive prototypes with variables, interactive form fields, and branching logic
Unmoderated testing tools
These tools allow users to provide structured input on their own time.
- Validate.ly — Unmoderated usability testing
- OptimalSort — Card sorting, can be done moderated or unmoderated
- Treejack — Tree testing, can be done moderated or unmoderated
Analysis and synthesis tools
We use a variety of tools to conduct analysis, synthesize findings, and create deliverables.
- Reframer — Tool for coding raw observations or support ticket data and creating themes
- Handrail — Notetakers input observations within a task or section, allowing for structured data
- Trello — Flexible tool to organize cards and lists; we’ve used it to build themes from observations
- Atlas.ti — Powerful qualitative data analysis tool for various sources, often used in academia for behavioral/verbal coding.
- Mural — Online whiteboarding: basic design functionality but allows users to edit a whiteboard without creating an account; we’ve used it for participatory design
- Miro — Online whiteboarding; has more design capabilities than Mural; we’ve used it for visualizing research findings
Limitations of remote research
With all that in mind, there are some drawbacks to remote research to consider.
Doing research remotely requires participants to use some specific technology. This can filter the participants in a way that may not be representative — i.e. the participants in the research may have more or different technical familiarity than the intended targets. Also, it can take a bit of time to guide participants through the setup.
Getting the team, including observers, on board with the research plan can be an additional challenge when they’re not all in the same room and can’t take non-verbal cues. Should observers chime in with questions during a session, or should they relay them to the researcher via a backchannel? What is most important for note takers to capture? Where should the notes go? We make a point to over communicate the most important instructions in email, in calendar events, in Slack right before we begin a session, and sometimes we even state them in the meeting when we begin.
Time zones can get mixed up! People are busy, and if we state a time zone and a given time for a session, the time zone is sometimes ignored. Calendar invitations and tools like Calendly that detect and confirm a person’s time zone before showing session times can help.
Don’t let the pandemic stop user insights
Remote research is a valuable tool for guiding your work toward outcomes and reducing wasted time and effort. At Ad Hoc, we collaborate with and listen to users, stakeholders, and partners. We also create straightforward systems and respect users’ time. When applied wisely, remote research allows us to better achieve these values, whether or not we’re dealing with the new remote reality of working in a pandemic.
