4 Lessons I Learned from Doing Usability Studies with International Audience
TL;DR: Over-preparation and building connections
Doing user research with international audience is both a challenge and a privilege. I’d like to share a few things I learned through my personal experiences in the past few months. I hope it proves helpful to anyone in the future in the same shoes.
1. Figure out logistics thoroughly and well in advance.
This should be a no brainer, but too often, I see people not planning enough or only do it at the last minute before the sessions start. Aside from your typical preparations like testing materials, multiple reminders, NDAs, and compensations, the long-distance nature of testing participants overseas requires more considerations: Can your participants access your app/website in their regions or countries? Can you find a place that’s quiet enough with a reasonably good signal (and are the labs available when doing tests out of regular hours in your participants’ time zone)? What are some of the local regulations to be aware of? Which currency should the compensation be provided in (and in what format)? What is the backup plan if something goes wrong? And does your Skype account have enough credits to last the whole session (yes, this too)? Basically, you need to be more prepared for this type of testing because there are more variables present. Testing is expensive both time-wise and money-wise. You don’t want to waste any of your testing sessions for being underprepared.
2. Explain “Think Aloud” at least twice. With Examples. That are relevant to your audience.
Actually, this applies to any research methods that your particular audience may find foreign. I mentioned the “Think Aloud” protocol because it is one of the more popular methods. During my testing with a culturally reserved audience, I had to repeatedly remind my participants how to “think aloud” throughout the session with examples that related to them and to encourage them to articulate their thoughts freely. This also implies that you as the researcher must know when to push and when to keep quiet. Relatedly, do look up how to say UX jargons like prototypes in the targeted language; otherwise, it could lead to inconsistency and inaccuracy (and makes your team look unprofessional). Fortunately, once you have learned how to handle these audience-specific cases, the research moves forward much more smoothly.
3. Hold on to your good participants! And ask them to refer candidates.
One particular challenge about testing users remotely is that you have few local resources for recruiting and screening participants. Besides the traditional recruitment venues, ask your participants to help when you see fit. Good participants invite good participants. And by good, I don’t mean people that only talk about good things about your product (obviously). I mean they fit in your target audience group well and are open to giving honest feedback. These participants could even become your first group of real users. Bonus brownie points!
4. Coach your colleagues on doing usability testing sessions.
This tip is especially important if you work at a smaller company, or if you are the first (and possibly the only) professionally trained UX researcher. Because of the time difference with the users and being Agile, my team usually schedules all tests on a weeknight for every sprint. Technically, I could have just increased the time I spent testing with users. But let’s be honest, doing 6 back-to-back sessions in one sitting isn’t most ideal scenario (they can run until after midnight). I realized this quickly a few rounds into the sprint. It was not sustainable or scalable. Upon noting this, I encouraged other teammates to practice conducting usability research, while I observed and gave suggestions when necessary. Sure, the training might take extra time, but it pays off in the long term. And the good thing is, usually there is already someone on your team that wants to talk with users directly or just learn more about user research. In my case, I was able to prepare a designer and a content strategist to lead the testing sessions a few times. On the one hand, I expedited the research process; on the other hand, I too grow as a researcher when I coach and guide new people.
Whew, that’s it for now! I’d love to hear what you think about these tips and would appreciate any additional ones! I will keep updating with the latest learning as I advance in my UX career.
P.S. Shout out to the Junior UX Community members for providing great feedback for this article.
