Educating User Researchers: Using UX Skills to Design a Research Training Program
Lindsey Latiolais presented “Educating User Researchers: Using UX Skills to Design a Research Training Program” at UX Camp Fall 2020. Enjoy!
Many people hoping to break into the field of user research don’t even know what skills are necessary to do the job and are missing fundamental concepts. They have to invest additional time and sometimes resources into gaining more skills, in addition to what they’ve already put into learning design skills they won’t use, and they have to rely on volunteer-mentors or companies to create apprenticeship training programs that vary wildly.
Some UX training programs for combine user research and UX design capabilities. This contributes to two problems: people aren’t appropriately trained in what they’ll truly need to do the work and to get a job in the field, and UX research isn’t respected as a discipline requiring specialized skills and knowledge. Training programs that focus on user research will give people the confidence to advocate for themselves and their expertise and, hopefully, encourage companies to recognize that not just anyone can whip out a survey or a usability test script and get back accurate, actionable insights that drive a better product.
Senior UX Researcher, Neighborhoods.com
Lindsey Latiolais is a 10-year veteran of UX research, currently working as the Head of Research (aka the only researcher) at Neighborhoods.com, a post-startup real estate company. She has an undergraduate degree in Psychology and a Master’s in Human-Centered Design and Engineering. She’s recently been turning her skills onto the discipline of UX research itself to find ways to increase her impact and evolve the field.
Outside of work, when not in the middle of a global pandemic, she loves indoor bouldering and scuba diving in the tropics. During a global pandemic, she plays a lot of Minecraft and hangs out with a lot of dogs.
The following transcript very likely contains typographical errors. Please forgive any mistakes!
Good morning, afternoon or evening. At bread that I am Lindsay lately, I’m gonna share my work on designing a curriculum for people who want to become user researchers. So I’m a senior user researcher and the only user researcher at neighborhood dot com. They’re a company of about 70 people or so. So relatively small. I have a dog coats him rapping from the previous UX camp home addition with his name tag. He’s been a huge boon during the pandemic as my constant companion, and I currently live in Central Oregon, I just moved so I can be closer to my family, so I can trade helping them out with dog sitting for some social contact. I lived in Chicago before this. So it was pretty locked down. So first, I wanna talk to how I learned, what I’ve learned to become what I hope is a pretty decent user researcher, so I’ll start with my formal education. I have a Bachelor’s in Psychology from Oregon University, that’s me in the sunglasses in Tamera. I switched from animal science in my sophomore year because it turns out that I am not very good at memorizing facts about animal reproduction in the psychology program, I learned a lot about how to do research and pitfalls of poorly designed studies, how they can negatively impact participants as well as bias your outcomes, a part of every psychology research paper is calling out potential issues and biases with the study itself, so the researchers doing that study have to consider those things, and that really helped create that mindset of evaluating and understanding those things as an integral part of research.
Then I kicked around, figured I wanted to go to grad school, but I wasn’t sure what in… So I attempted a few company he’s up in Seattle, including Microsoft and Starbucks.
While I was at Microsoft on one assignment in the legal billing department, I realized that someone had to make the software that I was using work the way that it did, every decision they made was a decision, and it was pretty terrible. And not at all the way that we needed it to work, and I figured there must be people who were trying to do that work better than I wanted to be able to help them do that, so I dug around a little and found the technical communication program at the University of Washington, but I didn’t apply yet, so I just kind of was looking into it, so then I was tempting at Starbucks, and one of the managers there was really great about really pushing people to plan out what they wanted to do with their careers, even, and especially if it wasn’t at Starbucks itself, and I really love that style, that sort of idea of helping people no matter where they ended up, and that push was really influential at getting me to apply to that technical communication program. So I got a master’s in humans in our design and engineering. I know I said it was special communication, but they actually changed the name while I was there, and you could pick which one you wanted, so I went with this one, it seemed like I would have to explain technical communication told too much.
So this was a great opportunity to get into the mindset of UX, learn the of common language, and we got to work with real life clients, we completely redesigned website for a bank in Hawaii. We got to work with Microsoft physio team to do usability testing. So that was amazing.
We also covered some more academic topics like rhetorical studies that may not have had as direct an impact on my work as a researcher, but they were still interesting and definitely valuable for a general sense. So on the job training, I’ve been working as a researcher for about 10 years now, and that’s where I picked up most of what I use in my day-to-day
To my first job. I was lucky enough to get a position as a usability researcher with our small consulting firm. I went to a job Bear and I got the interview by saying that I was most like Jordy Lafarge of all the crew members, didn’t start track the next generation. That picture is a view from the testing room at a marketing research recruiting agency that we use, so beautiful. Downtown Seattle on the gorgeous, gorgeous view. We did work on mobile devices for the equipment manufacturer, usability testing out-of-box experience, with this job, I really… This is where I really learned how to be a decent moderator from a co-worker who was an absolutely fantastic moderator, it was great to see him work, how he interacted with participants. We once had to dismiss a participant after just 10 minutes of a 90-minute session and she was struggling with the product so much, and he was able to really do that without making him feel any worse, he made her feel like her feedback was still valued, even though she felt like she had really failed at the tasks at this position, and I also learned that nobody read reports…
I uploaded a final report wrong and it was only the front page and no one noticed for several weeks until I actually caught the mistake when I went to download it for another project. So from there, I went on to contactors, which is a giant or a consulting company, this pictures from Tokyo where I was sent to do usability testing with Japanese insurance sales people, and I also have to do an ethnography at a couple of Starbucks locations there. And that was super cool. So this position, I learned a lot about dealing with high level executives and stakeholders, one of them rub the side of my freshly shaved head without asking at a conference, and I learned the importance of delivering very obvious wire frames to developers so they don’t build your very very ugly and dark mock-up for an app, PR, Paramount Pictures, we did a lot of heuristic reviews there, which was something I hadn’t done previously since he took a lot of our time, and so they were easier to sell the clients. Also, I learned you can compensate multi-millionaires with just the box of one dozen in some microbes for an hour of their time, and they will absolutely light up when you hand them the box.
So then I went to Orbitz, and that was my first in-house position, and there I learned about benchmarking with quantitative data, we would do competitive reviews using a tool that would measure number of clicks and time on site and compare it with our own… And I also learned the value of using clipped for having impact… Those work a lot better than a 20-page report. This picture is from the Chicago pirate where they do an entry every year. I also learned there that some companies don’t consider the emotional impact or the ethical implications of AB testing designs, the team that was in charge of the AB testing created a test and they couldn’t get the data for it, so they just made up some numbers than when the test was successful at driving sales of plan tickets and hotel rooms, possibly by increasing the user’s anxiety over a shortage, they just turned it down 100% with no plan or discussion of how to replace that big data with real data. So orbit was merged with Expedia, and so they laid off the researchers in Chicago, so I went and did a short contracting stint at EDTPA, which is pretty cool, ’cause now I get the house George Lucas name on my resume, and their website aimed at providing teaching resources to teachers of the K-12.
So here, my role was really evangelizing research to stakeholders to get their buy-in on that whole process and the tools to carry it out. I hadn’t had to do that in previous roles, I’d always had a research manager, this was my first dig of the solar researcher, but it was only a few months and I don’t have a picture from there from that time, so you get a little baby. Yoda, then I do Bradfield, a company of about 80 people at the time. Again, I was the sole researcher and I had one designer and then a few PMs, they’d never had a researcher before, so I had a very blank slate, and as the only person with experience in research, I was very much in charge of research operations for the entire organization, this was my first real exposure to running and analyzing A B-test myself, and then incorporating those into the larger research repertoire, I was able to do a lot more exploratory research here to build up a knowledge base around users outside of just task completion and usability analysis.
I was also trying to set up a good research repository because we were using Google Drive, and for anyone that uses the Google Drive, you know that information on Insights is just a lot more difficult to find after the fact, it’s a lot more difficult to reference past studies in that way. So I was working on that when I left and went to where I am now, which is neighborhoods dot com and… Not Nextdoor dot com, neighborhood dot com is a real estate website. I joined a team with several designers, which is nice, they had previously had a researcher, but it had been a while, and they didn’t have a good system for documentation, so research findings were all over. Some were in Google Drive, at least one was an envision, some of them I know have been completely lost because you kinda have to know where they are to find them, and unfortunately, all the people who know where they are have left… They
Do have confluent, which was mostly used by project managers when I joined to document kickoffs and retro, so I created a space there for documentation going forward.
I also have a large enough audience at neighborhood for insights.
So I’m able to use workshops as a delivery mechanism. When they’re appropriate, and my belief is that engaging with those insights to generate ideas help them stick a little better in people brains going forward, plus
I think workshop or super fun and a great way to exercise your creativity muscle, and we can involve people who may not normally be part of that ideation process, like project managers and engineers and help them feel a little more buy-in on the projects. So just to recap, this is how I really learned to actually be a user researcher.
I also do mentoring. I’ve been a mentor for a couple of years through UCAS mentorship program, UX coffee hours for the last few months, and I mentor start-ups through 1871 in Chicago. I didn’t have a ton of mentorship through my career manager at Starbucks, my teams lead at TCS somewhat, but I wanted to provide that for people who are more junior in the field who are looking for help with career advice, ways to have more impact or just people who wanna know what being a researcher was like and how to get a job doing it, so that where a lot of the sort of informal observations for this have come from, so now into the cruiser tin for this curriculum, essentially. So what really sparked this was that I was in a conference in June, UX research anywhere it was called. And there was a talk by Molly Stevens called escape velocity. And I’ll put a link to the Medium article she wrote in the slack. It’s a great… It’s a fantastic article, it’s about some of the issues she sees and user research and how we as researchers and help fix those in our own organizations, but that gave me the thought that many of the problems that I see and some that she pointed out could also be addressed with some sort of user research-focused training program, which doesn’t currently exist, instead of solving these issues as individuals or one by one, maybe we could change actually change the course of unordered by starting at the very beginning of people’s careers.
So what are some of these problems that I see? What is the design of a curriculum trying to solve specifically… So one is there is no existing training program.
The vast majority of what I learned has been on the job, and in my mentorship meetings and in some research-focused research or focused online spaces, I see a lot of questions come up that don’t really have good solid again, there like, How do I transition from user research from the XFL, or where do I go to get trained Avenue researcher and
Answering them one at a time, it feels very much like bailing a boat with a leakage current programs like group camp or even master’s degree programs or undergraduate degree programs really focus on training designers who might also have to do some research, not on training researchers themselves, which leads to the next problems.
There’s no shared knowledge among user researchers, especially with regards to epics and biased, I definitely see some potentially questionable research in Facebook groups or even in the news, they make it clear that not everyone has the same education level or viewpoint on these topics, for example, Facebook a few years ago, did some research on users without their knowledge or consent to see if they could negatively impact their mood, based on what they show them on their news feed, and it turns out that they can in fact do that… This is a picture from the Stanford Prison Experiment, which is pretty notorious in psychology for causing trauma to participants, they got a bunch of college students to pretend to be proven guards and prisoners, and those persons have been very much took on those roles in a lot of ways like Ontario covered a little book it in his book, ruined by design, when he talked about designers being held responsible for the damage done by things that they’ve designed, the Prema them out of the ones who need to step up and speak up. And user researchers are… Absolutely included in that call. But we may not have the knowledge and the tools and the language to do that.
I tried to go to a few conferences every year, and I’ve seen a lot of people talking about how to train other people in the company to do our jobs, to democratize our work to those people. I believe that is a terrible idea, we don’t ask them to jump in and start coding if we don’t have enough engineers, because we recognize that engineers have specific skills and training that other people at the company don’t have, and trying to use unskilled people could result in catastrophe, that brings down the whole website for a day or more, user researchers have even more responsibility than that an incorrect Insight could lead to months of wasted work. So how did I act? How I started designing this curriculum, so I created a design brief… What am I actually designing? What decisions am I making? So a big challenge is what topic should I cover and how deep should I go with them.
How much is enough that someone will be able to do the job of a user researcher without giving them too much information that’s gonna pick up too much time and money to teach, essentially. And then related to that, how long will it actually take to teach these topics? And then related to that, again, is this going to be a boot camp or a degree program of some kind is gonna be four weeks, 12 weeks. Two years, four years. This one is a concern to me because I want to make sure that any education I provide is gonna be accessible to as many people as possible, but that the people doing the work in teaching the content and creating the content are still being compensated for the work that they’re doing…
Although, people who wanna get into user research often already have a four-year degree or a four-year degree is required for the job, and I don’t wanna stack an additional cost on top of those people or take a long time to train them when they’re unable to work at the same time. A lot of people come into from other fields, and so they may have the loans they have to pay, they may even have kids that they’re responsible for, things like that.
So once that’s kind of decided, then the question is, what is the delivery mechanism is is gonna be pre-recorded videos, classes, will it be a single program or will I have multiple track or different types of researchers who might be teaching it? Well, I have all a cart or by the menu, you have to do the entire program, or can you just take bits and pieces of it one class at a time, and then what type of educations do people need to get into the program? Do they need to have a bachelor’s degree? It doesn’t need to be in a particular field, if it is in a peptic-LAR field, does that mean they get to skip some classes? So that’s kind of the design brief, so now what is my progress so far… I’ve done some informal interviews, I’ve talked with Mentees and networking contacts, just asking them about programs they’ve completed, what they felt was missing from any education they’ve had, whether they’ve covered topics like building rapport with interviewing these… Or research ethics, both things that I think could be taught to you, the research and researchers and should be to us.
So these are some of the topics that have come out of these interviews and my own experiences that I wanna make sure to cover, these aren’t necessarily based on the classes that would be taught, so some topics might be covered across classes, some classes… And they cover multiple topics, so when I mentioned ethics and research, so things like What are some previous problematic experiences in other disciplines or even in our own that have happened and how could they be avoided, but also how can research potentially help mitigate biases in technology or discover biases in technology. And I also wanna teach the responsibility of reporting findings accurately and completely, we… We do have a lot of power in our organizations with these findings and these insights that we’re discovering and reporting… So analysis of qualitative and quantitative data. This one I see a lot in those online patients I’ve mentioned, when people are asking and answering questions, I see a lot of just general questions about how do you analyze and code an interview, but I also see people who don’t know the difference between qualitative or categorical data that’s expressed as a number and actual quantitative data, those are sort of two different things, and we analyze them differently, and then I also wanna teach some basic statistics and statistical analysis.
I had to take quite a lot of it in my undergrad, and I don’t wanna do that too much to people, but at least things like p-values and confidence intervals and just at least some of the language that goes into that, and if people want to do become a quantitative researcher, then maybe going even further into that kind of analysis, so just research planning and methodologies, this also… This question also comes a lot up a lot in terms of what methodology to use in what situation and what is even available to them as a choice. I had someone in a group camp program tell me that they were taught that there’s no such thing as a research before you have a wire frame to usability test, and that just made me so to add to her Because exploratory research is some of my favorite research to do, I love doing at Porto research, it really… It gives you so much insight into people in their lives, and I just find it so interesting, research deliverables. So in my Master’s program, I feel like that’s the report or even the design, the final design was something that the only options for how to deliver things, and now I see mentees and also when I’m just saying portfolio reviews, where it seems like someone just made something because they were taught to make it as part of the UX process, like a persona or a journey map, and they haven’t really been able to actually link it to a particular outcome that they’re going for.
So teaching people that dress even 20-page reports, have very specific uses, and what those uses are and what situations that are appropriate for… So this is something that comes up a lot at conferences, of course, even this one, but going even beyond your research deliverables to how do you document finding so that they’re easy, easy to find later. How do you act as a repository of user knowledge for your team? I get that a lot. A lot of people coming to me and say, Hey, do we have any knowledge about X, Y, Z about our users? And then how to deal with stubborn stakeholders or stakeholders that are just really aren’t so bought into our research or the findings that you have, and just teaching people both lessons…
Cross-functional collaboration. We hear that a lot. I hear that a lot. I see that a lot in job postings, and I get asked that question a fair amount, even How do researchers work with designers specifically, what does that relationship even look like, and how do we work with data analyst, data scientist, if we’re not doing that quantitative research or quantitative analysis, how do we partner with people who do, but also how do we work with PMs, how do we work with engineers, what does the product life cycle even look like, that’s not necessarily covered in some program… Some programs do some programs not how to work with stakeholders. So working with C-level executives or clients, the people who wanna know what’s going on… But they aren’t part of the team working on the project.
I think the ability to communicate with them is definitely something that… A skill that can be taught absolutely. So common biases is research, so this comes up in mentorship meetings and in online spaces, people will ask, How do I ask… How do I design interview questions? How do I ask survey questions to avoid leaving someone on… And then I wanna teach people about doing pilot interviews or beta surveys just to sort of test the test, things like that, so just teaching people about what can go wrong in our research and how to mitigate those things, or how to work around them, how to moderate an interview, just things like how do you build rapport quickly with a present… What do you do? What do you do if you have a problem? Participant, so I had a guy just a few months ago, who actually made several racist comments during the interview, some directed at my PM who ultimately dropped out of the call because it made her uncomfortable, and me and the designer who was taking notes were very uncomfortable, but I didn’t know how to end the session, I didn’t know what language to use, especially to keep it safe for everyone involved, and so that’s something that I think we should be teaching.
Also, I realized a couple of years ago that the skills… I have a building report which we would interview candidates can also be used in a job interview when I’m interviewing for a position, and so after that, my job interviews… I don’t know if they got more successful, but they definitely got a little more relaxed and sort of conversational. Uncomfortable. So that was very nice.
How do critique… So I hear this comment now and then that, Oh, UX people are so great at critiquing and taking criticism because they all went art through art school and have all done it a million times, but that is definitely not always the case, particularly for researchers. They tend to come up through things like psychology or sociology, and so I’ve definitely myself had to learn how to effectively give a critique and take it constructively. We didn’t do much of that in my master’s program. Specifically.
This one, I’m not sure if I can account for it, but I’ve heard that… I’ve heard from people, and I experienced this myself, that early on, they don’t really know enough about being a designer versus a researcher to know which one they want to do, so those are kind of the topics that I’ve covered it so far, and now validating. So I’m doing a survey, which is currently in progress, and I’ll throw that in the Lachine as well to kind of validate those findings, so I’m asking backwards-looking questions about people’s past training, about current skills that they have and have developed, and then what they may wanna know about going forward. And
Also for experienced researchers, what surprised them when they joined the field, and then for people who are recent graduates or who want to be researchers, what do they wanna know and what have they already learned in any programs that they’ve gone through?
I also wanna do more formal interviews, so the sort of informal interviews or just anyone I could get my hands on, so for this, I wanna do a little more formal recruiting to fill in some of the gaps. Make sure I’m talking to a wide range of people. I also wanna talk to hiring managers and recruiters who I haven’t really spoken with much yet since they’re kind of on the other side of that table, hiring people. So I wanna know what are they looking for and what might be missing. And I also wanna talk to researchers in different but related fields, so business anthropologist, marketing researchers, etcetera, and find out about their training and what can we incorporate, and then we’re on to the actual design piece.
So I will then sort of… Flux is out a lot more. I may do experience in not being or user profile, if they end up being necessary, this is where I’ll involve some educators and other designers, if they’re willing to help out just to get some more eyeballs on what I’m creating, and then I may potentially create a data of the program, just to figure out how long it takes and if it’s even worthwhile.
And that is all of my time of… If you have any questions, you are welcome to email them to me or tossing the Slack channel, I will definitely jump over there and answer questions.