Tent Talks Featuring Alec Levin – Beyond the Interface: The New Frontier of User Research

[00:00:34] Chicago Camps: You’ve observed a significant shift in researcher activities, moving toward greater involvement at the executive level. Can you share some insights into what’s driving this change and how researchers can best prepare for these new challenges?

[00:00:48] Alec Levin: So first of all, it’s going from 0 percent to 10%, hopefully more.

And I expect there to be more soon. We’re not going from 50 percent to a hundred percent, just so we can level set there. Okay. I think there’s a bunch of things that are happening at the same time. And in organizations where they have a really well functioning insights departments, whether you call it the research team, whether you call it the insights team, ones that are functioning really well, there’s becoming more and more of a realization that these insights functions, these research teams can do a lot more than just make sure that your onboarding flow is an effective experience, let’s say.

And part of why this is happening, I think is. Twofold, one is there’s a lot of change and whenever you have a lot of change, you have the potential for the fundamental frameworks and lenses by which you view the success of your business can change, right? If you, obviously the top of mind example for everyone is this AI stuff, right?

If you are, for example, Getty images, right? Your entire business is instantly changed, right? With some of these new technology. How do you understand what that means? How do you understand what your portfolio of assets is good for versus not good for? They’re probably their entire product needs to be reworked.

And when you have this dramatic shift, having professionals who know how to investigate things, study things, both from a qualitative and quantitative point of view is a really effective thing. So obviously companies, you can put them on a spectrum from deeply affected by AI or any other change to not affected at all.

And those that are deeply effective will have a greater incentive and need for research to happen and for those insights to reach the executive level so that people can make really thoughtful decisions around how they’re planning for the years ahead. And I think the second thing is that there are more and more savvy research folks Who are understanding how to speak the language of a business, how to read a PNL, how to read public filings.

If they work at a large publicly traded company and understand what things matter to the business. And with that in mind, they can be much more effective at delivering work and projects that really. Support the bottom line and the growth initiatives that your businesses are focusing on. So those are a couple of reasons why we’re starting to see this go from very little research and insights happening directed towards the executive level.

Towards a small amount, but meaningful. Research is a bit of a different unique function in a business or organization, partially because it is, I think, the only function that creates value indirectly. There’s no other function in the business where your success is about helping other people make better decisions rather than producing something that’s valuable in and of itself.

Software engineers write code, marketers run campaigns, designers design experiences, researchers produce information and knowledge that hopefully allows others to be more effective doing whatever it is that they’re doing. And so the way we communicate. The words we use, all of this matters. So for me, I never use the word experiment, use the word study.

I like to use the word investigate. I like to, those types of that type of terminology helps people conjure up the right image of what the work is and what their expectations should be. So when you think about something like experiment, a lot of folks who may be a little bit more naive to the work might picture a white lab coat or a two way mirror, and that’s not really what the work is in 99 percent of the case.

I think it’s a part of it is cultural. I think there’s been a cultural desire or aversion to talking about money, talking about unit economics, talking about growth. The reality is that these, this form of commerce is extremely important to The way everything around us works and to be intentionally naive to it and to decide not to try and understand how all these individuals who are involved, especially the leadership level are making decisions, I think is like super, super foolish.

And I think it’s also easy to forget. There’s a lot of people who have this have negative feelings towards capitalism and that sort of stuff. And what I like to remind people is that it’s not just all these super wealthy people who are invested in these companies. It’s. They’re firefighters pension fund that are in, it’s your public sector employees, pension funds.

It is your grandma’s mutual fund that she’s been investing in the last 40 years. All of them are expecting to be able to take care of themselves and their families, especially in their older age, by putting their money to work in companies and businesses, like the ones that you work at. And so I think that there’s real moral value in doing great work and trying to create businesses that are more successful and the more you understand that language and the more you understand the decisions being made, the more you’ll be able to do that.

[00:05:51] Chicago Camps: With the rapid development of AI and large language models, how do you see these technologies fitting into the research landscape? Do you believe that they will compliment traditional research methods or could they potentially reshape our approach entirely?

[00:06:07] Alec Levin: That’s a great question. It’s very tempting to jump into this is the way it’s going to be. This is what I think, but in many of these cases, it’s very helpful to look at analogous situations especially in fields or situations that are completely different, right? I think these technologies are going to be adopted in mass. I think that in 20 years, there won’t be any researchers who aren’t using these technologies to do their jobs.

At the end of the day, there’s a very simple reality, which is when you have a new technology that can alleviate significant portions of the work, then there is going to be a tendency towards using that technology. And from an economic lens, it makes your labor less expensive and your output much higher.

And so those forces are very strong. Now, does it mean that the software that it is today is the right software? It does not mean that. Does it mean that this software is appropriate to use in all cases? In most cases? No, it does not mean that. Does it mean that we have a lot of work to be done in order to shape this software in order to be used in appropriate ways and to make sure that we’re not harming customers and stuff like that?

Absolutely. All that is true, but the forces here are too strong. And when we try and think about how do we know this, what’s the case? You can go back to look at like agriculture 500 years ago, a thousand years ago. Back in those days, you would have tons of human beings working on the farm, right? And if you look at the early days of America, something like 95 percent of everyone who lived in the 13 colonies back in those days worked on a farm.

And over time, now that number is about 3%. And how did we get there? First, you had the use of labor animals, and then you have the invention of the tractor and then the mesher and then the grain dryer and all these additional Instruments that were basically used to make the work more productive allowed people who used to be on the farm doing very low value work by producing a few bushels a day or whatever it is of wheat.

Now they can go and specialize in making shoes, right? Or creating or manufacturing pots or doing entertainment, right? That the proportion of human beings that are involved in the entertainment industry, it’s higher than it’s ever been, right? Trying to keep people happy and entertained and whatnot.

And that’s the result of us embracing technology to make all of us be able to produce more with less. And that I think is the type of analogy that is the most helpful for looking at this stuff. Research in particular has been really in many ways, not been very affected by the last 15, 20 years of software, right?

If you were doing ethnography in 1980. It’s not that different to, it hasn’t been that different up until very recently. And now with these new AI and large language model products, it has the potential to change it a lot because for the first time we can actually communicate and analyze itself and all that stuff.

When it comes to a lot of these employment things, right? It is true that sometimes that these new technologies change labor, the labor market, sometimes in a negative way, especially initially. But however, two things happen. One is you have some proportion of the people who stay in, in the doing that work produce a lot more and are able to typically earn a lot more because they’re able to produce a lot more.

The second thing that happens is the labor that’s no longer working in those roles can do other things. But after you look ahead a few years, a lot of times when you have this new technology the labor, the desire for labor for these roles can explode. And so if you look at things like how many data analysts existed.

In 1925, right? Probably very few. And then at some point along the lines, you have computers that can do this great math using this tool called Excel. And now you don’t need the IRS used to have these massive rooms filled with people doing calculation after calculation, all those jobs are gone.

However, we have more people doing that type of work than at any point in human history, by probably multiple orders of magnitude. So the technology also enables more and more people to do that work and do more and more stuff, but it takes a little bit of time to catch up sometimes, but it does mean there’s a brighter future where people who are right now doing basically making handmade ceramics, being able to all of a sudden mass produce incredibly beautiful pots and pans and whatever would be the analogy.

[00:10:33] Chicago Camps: There’s a potential future where designers do more research with the support of a centralized research function. What benefits do you see this model bringing to product development and how might organizations start to implement such a shift?

[00:10:46] Alec Levin: Yeah, look, I think there is a tendency for a lot, among a lot of folks in the research community to want to hold on to research, right?

This is our craft. We should be the ones doing it. And I understand that. Again, one of the things that makes research a little bit different from a lot of other functions is research is something that everybody does on a day to day basis, right? You try and figure out where you want to go out for dinner with your friends.

Like you’re going on Yelp, you’re reading reviews. You’re looking at the numbers, that’s a form of research. It’s a different form of research, but it’s a form of research, right? And it’s one of the things that kind of makes the role unique. And I think there’s a fear that in general, that the more that we give away of the work the less influence that we have, the less we will be valued.

And I think that’s just not true. I think that the more that we’re able to focus on uniquely challenging hard work, where we have our deliberate, really high leverage insights, the more powerful it’s going to be and the more people are going to be excited to, to engage with professional dedicated researchers and incised professionals.

So I think the other thing there too, that’s important to keep in mind is there’s a cost to having research, right? A communication cost. And when you’re sitting as a researcher and you’re interviewing customers, you’re interviewing users. You then have to go and translate the things that we learn and see into the minds of people who are then going to make choices that own that product, that own that experience, that own that whatever.

And there’s things that are going to be lost in translation. There’s going to be things that you don’t notice that someone else would notice and see as a moment of inspiration because they do the design work or something that you see somebody notice or say where product lead would say, Oh my God, that’s a really interesting opportunity for monetization.

And that won’t translate always, and there’s work to do that communication overhead. And so when you give away parts of the research where folks are going to be able to do it competently, it’s, of course, there’s some downside in you’re not having a professional to the work, but there’s also upside to there’s upside and there’s less work on the communication side.

And there are certain things that you won’t notice that are important. That’s somebody with a more complete amount of context, because they own it would notice. Some of the most effective research teams that, that I know of function this way. They have resources, training and programs to support non researchers to be able to do especially some amount of tactical research work around the usability testing or concept testing, for example and then they focus on some extremely high leverage research initiatives, often in partnership with the executive team, right?

For example, we had a talk at UXR com years ago from the team at Slack that they were trying to expand into Japan, right? And so obviously people could use the product in Japan, but it wasn’t really made for Japanese characters and Japanese business culture. And there was a lot that they wanted to change to make it really take off in Japan, especially with like non kind of American businesses that were operating there.

And the research team was instrumental in making that product work, right? Because the way people communicate is different. You would never at channel when your boss in the Slack channel and your boss’s boss in the Slack channel, if you were an individual contributor in Japan, it’s just not the way things work.

Of course, the syntax is different, but even when it came to launching the product in Japan, the research team was key to writing the speech the CEO gave, right? Again, understanding the nuances among how people collaborate, how people communicate and what the business culture is there. So all that is, that’s a great example of a really high leverage, longitudinal research initiative that involved multiple different specialties of research, a ton of planning and that it was worked, that was developed very closely with a priority of the executive team, which is expand to a new market that we don’t understand yet.

And I think that when research is focused on those types of problems, it’s in a really good place. I’m like a huge advocate of non researchers like watching the research sessions and being involved in some of our sponsor companies. Like LookBack are really great tools for that.

But again, a lot of these conversations around things like democratization are really focused on the negatives or the positives, depending on how you feel. And what we really need to be doing is analyzing trade offs and trying to think about in which situations this would be appropriate versus not appropriate and which situations do these trade offs skew towards non researchers doing the work versus researchers doing the work. And I think that’s a way healthier way to look at a lot of this conversation around that.

[00:15:30] Chicago Camps: As the CEO and co founder of Learners, you’re at the forefront of identifying growth opportunities for user researchers. What gaps in the market did you see that led to the creation of Learners and how has the platform evolved to meet the needs of the research community?

[00:15:46] Alec Levin: We’re continuously learning about this. I think first of all, there’s tons of growth opportunities for researchers in general. Some of them we’ve talked about around literacy on the business side, around leveraging new technology like AI for sure. But there’s also a ton of stuff that’s like way more niche.

That there are some researchers that are working on mixed reality experiences and inventing new research methods around that. There’s researchers who are working with physical products and hardware, and those are different types of experiences and different types of things to work with.

There’s so many interesting things that are happening. And part of what we started learners, we just said. We’ve got to find ways to make the professional development learning and the growth that people want, how to make that accessible to everybody. And originally we were just running this conference, UXRConf.

And then when the pandemic happened, we moved it online. And when we did that, the unit economics in terms of like cost of running it changed because we’re not renting giant venue halls anymore, and we’re not paying for catering. And we saw so many more people. Attend remotely when we reduce the cost by 90%.

And we said, there’s clearly something here. And we’ve learned a lot about what it’s going to take to build like a. Kind of learning companion that can follow you throughout your career. And some of the things that, that I can point to, one of them is that community is super important, especially community, people around you.

Whatever reason events are really important from a learning point of view. We’ve, we’re tried to focus more on the software side and the event stuff just keeps pulling us back. Cause that’s one of the things that people want. People’s learning needs change a lot. That’s one of the things that makes building in the ed tech space really hard is a lot of your successful customers churn, right?

If you finish the course and your customers are really happy, they got the value out of it and then they leave, right? That’s not the case. If you have a really great SaaS product, happy customers stick around a long time. And so there’s a lot of really unique challenges about the learning space. But the thing that ultimately we’re really excited about and bullish about is that I think the more that we create opportunities for new people.

Working in different locations and different types of things to be able to work hard on their stuff, bring their head up for air, have a hand them a mic, give them a stage and say, share what you’ve learned. What are some of the key takeaways? What are the things that other people can benefit from? That’s where you’re going to start to see the acceleration of the growth of our field and when we extend it to other fields, similar things, because there’s so many amazing people doing like groundbreaking work that are just too busy to figure out how to build an audience and figure out how to market their learnings and it’s just too much work.

So if we can figure that part out, all that infrastructure. I think we’ll be in a much better place. And that’s like our vision for what learners should be.

[00:18:33] Chicago Camps: What emerging trends or technologies do you believe will have the most significant impact on user research in the next few years? How should researchers and companies adapt to stay ahead?

[00:18:46] Alec Levin: It’s hard to look away from the AI stuff. I think it’s going to be massive for research. I think one of the big things that’s held research back is that there’s not a way for non researchers to play with the data and to see for themselves and manipulate the work that we do. There’s a PDF output and that’s it. But with some of the, what I expect will be developed over the next couple of years from, in terms of research tooling that leverages this AI stuff is.

The ability for where, whether they’re executives or people on other team to like basically go ahead and query huge numbers of transcripts and interviews and see like when people are talking about the food that they eat, what are the types of ways they describe it? And to have a, some kind of chart just pop up and here are the types of adjectives and verbs that they use around preparing this, and that’s never been possible before, right? Even if you have a repository or you have a drive folder with a bunch of work that you’ve done in a bunch of presentations, you just have to go one by one. And so I think that’ll be a huge change for the research space and will really help other non research functions see more of the value that comes out of that.

You can track these things over time. You can manipulate different questions. And I think that the AI stuff is going to unlock a lot of that. So I’m really excited about that.