Tent Talks Featuring Bryce Johnson: Innovation Without Barriers: Behind the Scenes at Microsoft’s Inclusive Tech Lab

[00:00:36] Chicago Camps:  Bryce, would you show us around and tell us about the Inclusive Tech Lab?

[00:00:40] Bryce Johnson: Yeah, I’m in the Microsoft Inclusive Tech Lab right now is our facility here on the Microsoft campus. The Microsoft campus is one of the largest corporate campuses in the U S. So it’s important for us to have a facility where we can engage the disability community. This is the second version of this lab.

Our first one was a little scrappy. We built it in a basement, hosted over eight thousand folks, but this one’s been open for about two years and we’ve got. A lot of natural light, I’ll do a little quick spin,

A lot of assistive technology. If you’re wondering what that is, that’s our sensory room. Do I know what I’m going to do with the sensory room for technology? Not really, but I do to have one sensory rooms are really important when we engage the neural diverse community, thinking about how people get their sensory needs met.

That’s why we have one. We’ve hosted over sixteen thousand folks in our five years. But the primary reason why we’re here is that is to bring people with disabilities onto the campus to work with us through our inclusive design practice. We believe in nothing about us without us. It was great for us to be able to build a room that was specifically meant for the disabled community.

So when our colleagues come visit, even though we call it a lab, it’s not like a usability lab, there’s no two way glass, there’s nothing like that. This place is for the disabled community and our colleagues are the ones that are visiting when they’re here.

We’re not therapists. I don’t ever claim to be a therapist. I work with a lot of therapists. This isn’t necessarily a practicing sensory room or Snoezelen room if you’re a therapist. We do have a lot of good kind of equipment in there. And really you just need to have a lot of variety of things that people can use to get their sensory needs met.

I have some of the auditory stuff turned off. It gets a little bit noisy in here when the bubble machines are going, but like the flashing lights and the proprioceptive kind of thing. And because we’re Microsoft, we have a Minecraft theme. So I’ve got my custom creeper blanket or a blanket and rug.

[00:02:50] Chicago Camps: Of all the projects you’ve worked on at Microsoft’s Inclusive Tech Lab, which do you believe has made the most significant impact on making technology accessible to people with disabilities? Can you share a story about how it’s changing lives?

[00:03:05] Bryce Johnson: Yeah, our work here really started with the adaptive controller. Most people recognize it from, we have this Super Bowl ad in 2019 for Microsoft, and I think a lot of people who know this work could be familiar with this ad. I think what’s really interesting though, is that we would get a lot of people that would be like, Hey, it’s great that you made a controller for disabled kids.

And we didn’t make a controller for disabled kids. Very few things that Xbox are for children. So you talk about being the demographic of gamers is very largely people like us. It’s actually quite huge. We actually started with Vets. So this was a video from 2015, that’s Sergeant Josh Price. And we met him with Ken Jones from a charity called Warfighter Engage.

And basically we had to recognize that our traditional game controller, our beloved Xbox controller unintentionally excluded people through its design, right? This thing assumes that you have two hands to hold it, that you have two thumbs for the sticks, that you have a fluid range of motion, you have the reach with your index fingers to get to these bumpers and triggers, and that you have the strength and the endurance to hold.

And what we had to recognize way back then was that if people couldn’t use this, they couldn’t play Xbox. So when we would talk to the disabled community a lot, They would say things like, I play PC, which is a very normal thing to say in the gaming world. So you don’t need to think twice about it until like you really dig into it.

And they were really telling us, they play PC because this is right. So we started the adaptive controller with Josh back in 2015. And I will say that when you talk to like, when you talk to vets, this was at a time when we were still really fighting to get accessibility. Yeah. Beyond any kind of compliant, like into our product.

And because we were working with vets, this project, everyone knew we had to do it. It was really just how we went about it doing it. There was no arguing why we needed to do it. If you’re a vet and you grew up playing video games, if you’ve been in service for the past 20 years, you grew up playing video when you’re deployed.

That’s the primary aspect of your social life. It is hard enough for that to come home and integrate back into society. And if you have been injured in service and you come home and you can’t even play video games with like your crew, it’s really isolating. It is awful for people’s mental health. So we heard those stories again and again.

And we had vets on our team, guys like Todd here. Todd’s a quad amputee. We would talk to guys like Todd. And so it really just became really important for us to figure out how we would go about it. And there’s lots of different prototypes that kind of happened Before we, we did this through two hackathons.

This was my hackathon project in 2016. I’m one of the inventors of the X Box adaptive controller. There are dozens of us, but I get to talk about it the most, probably. Still to this day, this was five, six years ago. And yeah, it’s been an important project for us. It, we built the lab to show people what, why we needed the adaptive controller.

Because slideshows and videos in meetings were cutting it. The, there’s that old saying in design, you’re not the customer, right? Here’s the problem in Xbox, by every metric that Xbox has, our employees are the best customers. They’re the whales of our industry. They play the most, they buy the most, they use the most stuff.

You can’t tell them they’re not the customer. They know they’re the customer. What you have to start doing is having a conversation around, you’re not the only customer. And that’s where we started the lab and got going with all that. I’ve been in accessibility a long time. This was definitely a huge sort of shifting point for me.

I’d worked on Xbox software for many years. All of a sudden I’m doing hardware. I’ve been in, yeah. And then since the adaptive controller, I’ve been doing hardware and now I’m shifting back to software because we’re really focused on it. It literally probably wouldn’t have happened if it wasn’t veteran.

And when Satya, the first time I met Satya, he walked right up to us. We were showing the controller and he said, I want one of these in every VA. And we were like, yes, boss. All right. So that’s what we did.

[00:07:30] Chicago Camps: Do you have any statistics or information about how many folks purchase the inclusive controller who are not in the disabled community?

[00:07:40] Bryce Johnson: I don’t really, and I think it’s an Xbox controller. So I think you’re going to get a certain amount of collectors that like buy them. I will say that one of the grand privileges I have here in my practice is that no one’s ever really plugging me for like sales numbers. Someone collects them for the longest time.

I don’t really even pay attention. It doesn’t really drive what we do. We have a mission at Microsoft to empower every person and organization on the planet to achieve more. And that mission basically drives. Our need to do this. We have a responsibility, we’re one of the biggest company in the world.

We have a responsibility to, to make sure that we’re including people with disabilities in the products that we create. We’ve certainly seen a lot of fun stuff. I will say, I think early on, I was really hoping that a lot of that stuff would happen, but the way that my work has evolved and the way that my focus has evolved.

I don’t really pay a lot of attention to non disabled people, so it could be, there could be lots of stuff that’s happening, but I’m much more focused on basically serving the needs of the people that we’re trying to identify. And we bring folks in here. That’s the whole reason for this place. I think one of the things about the lab that I think is really interesting, we host, we’ve hosted sixteen thousand people.

A lot of them are just people who want to visit. But we kick people out. We prioritize people who have disabilities in here. We told Walmart, they couldn’t come in because we had a transition program coming in. And that’s nothing against Walmart. It’s not them in particular. It’s just, that’s our priorities, right?

We have the disabled community take priority.

[00:09:13] Chicago Camps: What’s one of the biggest challenges you face in designing technology for inclusivity? And how do you and your team overcome it?

[00:09:21] Bryce Johnson: I have the challenge… I don’t know if I have the overcome it yet. So have you ever heard of the idea of functional fixedness? So it’s this idea that if I handed you a hammer and I said, this was a can opener, you’d be like, what? No, this is a hammer. It’s not a can opener. Or if I handed you a pair of scissors and said it was a knife, you’d be confused because objects have in our society, they have basically, they have need, right?

So with the controller, the hardest part that we have, we worked with a lot of disabled folks to optimize the device. And we worked with people at the time who really wanted to game. So they would go on that journey with us. But then you go out into the world and you go, Hey, this little square box is like a controller and everyone’s no game controllers look like, they don’t look like that.

They look like something else. And so that awareness of assistive technology is really, I feel like the biggest problem, followed very closely by stigma, right? So when I say stigma, glasses are assistive technology. When you and I were kids, we got made fun of for wearing glasses. Now that’s a ridiculous idea.

Everyone wears glasses as a fashion accessory, but it would the same thing be said about canes or hearing aids. We all know someone who could use a hearing aid that I don’t need a hearing aid. But yes, they do. So that stigma is, I think, a really important challenge when it comes to assistive technology.

But awareness is really tricky too, because I show up in, and this is one of the follow ups to the adaptive controller that came out last year. This is us doing this for PCs. This is a switch interface, a hub device, and this will do anything that a mouse or a keyboard can do. And I show up and basically go what’s this?

And I go, it’s a mouse and keyboard. And everyone’s like scratching their head going, I don’t understand. Which I get, right? We created a series of devices. There’s basically the hub and then there’s a specialty mouse. And then we have these buttons that, and like I said, these two, this will do anything a mouse or keyboard can do.

These can do anything a mouse or keyboard can do. Oops. The mouse is pretty much a mouse though, though that one is meant to be, is meant to be very familiar. So we want to always try to be as familiar as we can. Because I think one of the challenges in assistive technology is when things get unfamiliar and it’s something we all do.

If you’ve ever heard someone use a screen reader as a sighted person, you probably just think it’s like totally wild. You’re probably like, I don’t understand what’s going on here. You can’t wrap your head around how people are actually using it. And that’s because it’s been optimized for a very specific kind of community and use case.

And that optimization again is good. Like people need it, but we do have to recognize when we over optimize and we unintentionally exclude people. I think that’s one of the things we’re grappling with now is this awareness stigma complexity angle. I have this thing over here. I don’t know if you guys can read that on the screen.

But one of the things I like to tell people is you can do anything in Windows with a mouse. We’re always trying to design new ways to use it. I have to remind people it’s super important, you can do anything in Windows with a mouse. I have to remind people that and over here I’ve got a bunch of different mice. I’ve got, this is a mouth mouse. This is a mouse for people who might have cerebral palsy.

They don’t have a lot of fine motor control. They might have a tremor. That’s what this is for. This is a repetitive stress mouse, like a carpal tunnel mouse. This is a different type of carpal tunnel mouse. This is a head mounted mouse. You put this on your head and wear it like glasses. I’ve got mic mice.

I’ve got camera mice. I have all kinds of mice. I even have a shelf behind me up there. Wait, right there. That has a bunch of different mice on it. I have tons of different mice. Just to remind people that you can do anything in Windows with a mouse. So sometimes it is about designing the experience and optimizing the experience for a specific type of disability.

And sometimes it’s just designing a more accessible mouse.

Yeah. And and so what we do with our adaptive accessories is basically give people, I think, for people with limited mobility, the challenges is that doing like a three key keyboard shortcut for a lot of people with limited mobility is really impossible. And modern software has It’s just filled with three key keyboard shortcuts.

But if I can take that three key keyboard shortcut and put it on a foot pedal, I can put it on a head switch. I can accelerate the interaction of someone with limited mobility and allowed them to use their computer more efficiently.

[00:13:59] Chicago Camps: How do you envision the future of inclusive technology evolving? And what role do you see AI playing in making tech more accessible?

[00:14:07] Bryce Johnson: So I’ve recently published a guide that basically talks about how we think about it in hardware. I think one of the things in software that we’ve thought about for a really long time, software only has to have form because developers and designers don’t have infinite amounts of time, right?

Software can have any form we can imagine. So there is an old sort of joke around here that if a develop, if an engineer tells you something’s impossible, what they’re really saying is they don’t have enough time, because you can do anything in software. When I moved to hardware, I, however, I did learn about, and sometimes when a mechanical engineer tells you something’s impossible, they mean physics.

What we come up with is this notion, or this idea, when we started our inclusive design methodology, we really didn’t, we weren’t striving for this idea of one size fits all. We really wanted to think about fit. We wanted to think about one size fits one, right? I think something I say in here all the time that I really believe that all accessibility is personalization that takes into account human diversity, right?

That’s all it really is. How do we make sure that we’re taking into account human diversity? If you even look at something like as simple as responsive design, right? We always talked about responsive design when it came out. As the design fits the device, but I will tell you my phone computer and my PC computer don’t care about user interface cares about user interface.

People do right. So that changing to fit the situation is accessibility. It’s just accessibility for the vast amount of people. So how do you take those ideas of responsive design, that type of flexibility? And how do you really start to design interfaces where one size fits one? I think when it comes to AI, we’re going to get to a place really pretty quickly.

I’m also at Microsoft. So take this with a grain of salt. I’ve never thought of a time when the whole way that people can interact with computers can just be different for everybody. We really can get to a place when we, where we can see the fact that the way that the reason why interfaces will have, will be consistent will be because people need to learn them and trade knowledge about them.

Not because an engineer doesn’t have enough time to build a million different interfaces. So that type of shared knowledge is like a society, I think is what will keep us consistent. But yeah, that’s where it gets really exciting for us. This whole idea of multimodal input and output, I was on the launch team for Xbox One.

It was a time when I think people weren’t ready for what we were pitching with Xbox One. We had Kinect, which was a camera that was always attached. And you could go through the home screen, the home layout, the OS of the device. And you could do it with controller. You could do it with Harry Potter gestures if you wanted to, or you could do it with voice.

And you didn’t have to switch. You didn’t have to change anything in the flow. You could switch mode, input modality as you went through the flow. We talked about this idea of no cliffs and we designed it that way. And so from an input perspective, that was always really exciting. And that was a decade ago that we did that.

So when you think about MR and you think about what happens with, if I’m doing a Teams call in my office and I move into the kitchen, how do I transform that Teams call and the content on that Teams call so that I can still be present and part of the meeting while I’m like in my kitchen and the only thing I have in there is maybe a an Alexa echo, like an Amazon Echo speaker.

How do we transform the input and the output of that meeting? To the situation, to the environment, to the capacities that I’m in. And thinking about it like that in terms of situation is helpful for folks who aren’t in a disability. But for me, having a voice controlled, audio only experience would be great for blind folks, right?

Having a voice controlled experience is great for folks with limited mobility. So this type of morphing interface and this idea of multimodal input and output is really exciting. And I think We’re almost at a point where we can see all the input all the time. But to go back to that awareness thing, we almost have to limit ourselves because it could get too strange for people to quit responsive design and design systems didn’t replace design, but it just changed how we thought about it and it’ll change again.

One of my favorite examples recently from this idea of transformation is I was talking to a woman who works in operations on a military base and she’s autistic. And what she needs to get. From an information point of view, she needs context. She needs analogy. She needs narratives. She doesn’t do well with tabular data and bullet point.

And she works in the military, which is all tabular data and bullet point. So this idea of being able to take in content and tell like a generative AI, just like a large language model, Hey, put this in context for me, tell me a story. It doesn’t even really matter if it’s a true story. I just need this information presented to me in a way that I can consume it is really powerful.

[00:19:11] Chicago Camps: For individuals and companies looking to make their technology more inclusive, what foundational advice would you give them to start on the right path?

[00:19:20] Bryce Johnson: Yeah, the first and foremost one is this idea that the disabled community called nothing about us without us. Our inclusive design methodology strives to design with and not for. And that’s the way that we work. I don’t want to pretend that we crack the code. We’re always learning. But most of the time students will come up to me all the time and they’re really interested in this work.

And they’re like, I want to design this thing for deaf people. And I’m like do you know any deaf people? And they’re like, no. And I go, I figured out your first problem. We have to design with the people that we’re designing for. So that principle of nothing about us without us, very crucial.

And sometimes I just don’t really, I don’t waver on it. I don’t want to, I don’t want to tell people go read these books or go read these articles or go watch these YouTube videos. All of that is decent advice, but it never takes the place of actually engaging with the disabled community.

So that’s for me, that fundamental piece is to engage the disabled community. And yes, that can be hard and yes, people want to get scientific. User researchers quite often get way too scientific when engaging the disabled community for me. They try to get medical scientists, like medical stuff. And I’m like, this isn’t a clinical trial.

We really just encourage people to participate in disabled communities. And I think. You really do need to engage. When we first started thinking about our diversity inclusion practice in xBox many years ago. If you think about diversity inclusion in the vast majority of video game fields, it’s really about the representational aspects of diversity inclusion, largely versus the functional aspect.

So what I mean by that is I don’t see my race, my gender, my sexuality. I don’t see that represented in a game, right now. So we would go out to the disabled communities and we would talk about representation and quite loud and clear from the disabled communities would be like, Hey man, I can’t even play this game.

Let’s fix that first before we like worry about representation. And that was years ago. I don’t want to pretend that is how people feel today, but it was pretty humbling back then to be like, yeah, we’ve got a lot of things to fix before we can even get to that stuff.

[00:21:36] Chicago Camps: And now a question from our live studio audience, Kevin asks, “Has Bryce optimized that sensory space over time, and if so, how did you go about it?”

[00:21:46] Bryce Johnson: I wish I had a better answer here. So yes, I’ve optimized over the time. I buy stuff all the time. Again, I’m not a therapist. So we see how kids in here react, but we also are trying to think about how, what we can do in this space. So what I mean by that is we started our sensory room because you think about the transition activities that neurodiverse children have when they go from one activity to another, we would see sensory rooms in school, because kids needed them to get their sensory needs met so they could have the attention to focus in class.

They transitioned from home to school. That caused sensory stimulation overload. They needed to get those sensory needs met to achieve balance so they could focus in class. But we weren’t seeing it in reverse. We weren’t seeing that ability to get their sensory needs met when they came home from school.

Trying to act normal all day. And then they’d come home from school and they didn’t have their sensory needs met and they would do some pretty extreme things. We first started thinking about this from a point of view of like, how could people have sensory rooms in their homes? How could we help facilitate that?

At the time, weighted blankets were coming into homes, beanbag chairs were everywhere, everyone’s got plushies, fidget toys are now really common. Most of the stuff that we buy now, we just buy from Amazon. Cause while we do have some actual therapeutic equipment in here, there is just so much that you can buy out in the world I think that was the first version of our sensory space and we know people like it.

We still think about it a lot, but again I’m not in here talking about proprioception for kids with autism or compression for kids with autism or sensory stimulation for adults with dementia, I’m not. Thinking about those as a therapist, I’m trying to think about this as a product maker. So one of the things that I’ve been thinking a lot about, and this is super wet paint, I don’t want to pretend like this is like anything, but like a total musing on my part.

If you think about the pandemic, we recognize the importance of mental health. And when I think about the devices and the objects that we carry with us everywhere, I ask a question of what if the objects that are like with me all I’m due for more for me, for my sensory. Need, then just allow me to watch cat videos on them.

What if I could achieve balance, like with the actual object? And so if you think about the evolution of computing devices, it’s really easy to see that computers used to have the aesthetics of tools. They were a beige box. They sat beside your desk. Apple made these things into these objects of desire, of luxury, these things that say something about me.

But when I think about like. Where this evolution could go, it’s, what are the things that actually make me feel comfort? The idea of a luxury watch and the idea of a luxury blanket have two different attributes. And so I wonder about, could we start to imbue some of our devices with sensory stimulation?

In them, there’s a, something called com strips that are pretty common. If people are familiar with, these are, these things are called com strips. You can buy them on Amazon. They basically have a little bit of a texture. You rub them. These are knockoff commscripts, but these types of things are meant for you to rub and use.

We’ll get kids in here all the time that’ll grab this thing and they’ll be like, how do you use this thing? And we’re like, you’re doing it. It, sometimes it doesn’t have to be that complicated. I think that’s one of the things that people find really surprising about our work in here is everyone comes in here and expects me to be Elon Musk and talking about brain computer interfaces.

And then I show up with a pack of stickers and weighted blankets. And there is lots that we can do in this space to make people feel better. So I hope that answers your question about the sensory space. I wish I had this really like firm plan about it, but to be honest, we get enough value of just having the space in the room, people enjoy it and that in itself makes it earn its keep.

So any insight that we can glean about how we think about the evolution of the object from a sensory perspective is just gravy.

[00:25:47] Chicago Camps: And another question from our live studio audience. Yunke asks, “I was wondering if the next version of XAG will include more updates specifically for visually impaired users, such as audio descriptions?”

[00:26:02] Bryce Johnson: It’s interesting that you say that it’s I can understand what you say. So the exegs, those are some guidelines that we create for game developers Xbox accessibility guidelines and it’s interesting, right? Because think it’s funny to me as someone who’s worked in tech accessibility, the focus that we have on blindness and tech accessibility overall is like overwhelming it.

The vast majority of the work is blindness, so I do find it amusing when people think that like gaming is focused largely on mobility, which it has, and you’re welcome because I’ve been a big part of that, you’re absolutely right about audio description, but what I think is really interesting about audio description In general, is that you have a community of blind users who I’ve experienced largely are trying to get audio description as it was meant to be implemented in film and television in different realms, and I don’t know if it applies the same way.

So what I mean by that let’s put aside gaming for a second. The way that you would audio describe a Netflix show. Yeah, Is different than you would audio describe a tick tock video, right? And so what I keep seeing from the community is this desire to go. How do you describe your tick tock videos like TV shows?

And you’re like, but that’s not what that is. Right? And so as these like media forms evolve. I think it’s, I think it’s really interesting. I’ve definitely had friends who, who are blind, who basically say who, who are trying to shove audio descriptions and games in the moments where someone’s not speaking during various parts of the game.

I’m like, I think. I think we can talk about the modalities of the game and we can talk about yeah, maybe the game can pause for a second while it tells you what’s happening, right? So I think there’s a lot that we can do as a community to really embrace audio description. And I think what I find fascinating about audio description in general is that we’re grappling with this, like we’re still trying to get movies and TV, which are.

Up to speed, which is something that had been known for a really long time. And then you have a generation of visual communicators that are going to enter the workforce making TikTok videos, and that will definitely alter their aesthetic and how they think about presenting information.

I do think that there is a lot that we can do in audio description and, games are just one part of it, but it is an important part of it. ​