Why Gamers Paved the Way for AI | Databricks' Carly Taylor

Conor Bronsdon 0:00 Honestly, I'm just struggling right now because I I am in my head now picturing Bowser with a French accent, and I'm gonna have to go find this after the show. So

Databricks' Carly Taylor 0:08 I just made it up, but that would be hilarious.

Conor Bronsdon 0:20 Welcome back to Chain of Thought, everyone. I am your host, Conor Bronson. And today, I'm joined by Carly Taylor. Carly is the field CTO for gaming at Databricks and founder of g g AI. Carly, welcome to the show. It's great to have you here. Thank you. It's so good to be here. It's really honestly exciting for me to have this conversation because similar to you, I have always been fascinated by the field of gaming,

Conor Bronsdon 0:44 both tabletop and digital. And as you've noted to me before, gaming has driven the advancement of GPUs, graphical processing units, and it's really the tech that has built the current AI boom as we understand it today. I know most people understand at least a little bit of this story, and I'll recommend the incredible acquired episodes on NVIDIA if anyone wants to go much deeper,

Conor Bronsdon 1:10 from a podcast format. But can you start by connecting the dots a bit for our audience? How did the video game industry enable today's AI boom? Yeah. Absolutely. I mean, I could talk on this for an hour, and I actually have before at conferences

Databricks' Carly Taylor 1:25 to the the nerds who are interested in things like this. But, you know, it really goes back to even before the first GPU hit the market, which was invented by NVIDIA about twenty five years ago, gamers have been pushing the limits of what their computers can do since we started making video games on computers. Right? I think back to Roller Coaster Tycoon, which was written by

Databricks' Carly Taylor 1:50 Chris Sawyer, I think is his name. For the record, one of the best games ever. One of the best games of all time, still. And it has staying power. Right? And what's crazy

Conor Bronsdon 1:58 oh, sorry. Go ahead. No. I was just gonna say I picked it up on Steam recently and played it for a day, and it's just like, yes. Still good. Nostalgic. It's still fun. Yeah. What's crazy about that game is that he wrote that entirely by himself in Assembly.

Databricks' Carly Taylor 2:12 And Wow. The reason that he did that and anyone anyone who programs will know, like, Assembly is, like, the nerdiest of nerds languages. You know, you're basically talking straight to the to the machine at that point. And the reason he did that was because the CPU at, like, power at the time couldn't handle the types of things he wanted to do with, like, guest interactions and the way that he wanted this game to work. So in order to hyper optimize this game, he just wrote, like, pure machine level code so that he didn't have to deal with any abstractions or anything that would slow down, like, what he was trying to build.

Databricks' Carly Taylor 2:45 And I think that kind of encapsulates the gamer mindset, which is, like, your only limit is your imagination in what you can build, and they won't let things like compute power slow them down. And so I think that, you know, that mindset set up a trajectory for gamers to continuously push the limits of what compute could do. When the first GPU hit the market, like I said about twenty five years ago,

Databricks' Carly Taylor 3:07 I think it was called the GeForce two fifty six. It had the processing power of, a potato nowadays. Right? Like, it Yeah. It was very rudimentary from what we consider today. But over time, as GPUs started to become more used in gaming, you know, they really accelerated graphics processing and the ability to make worlds richer and look more vibrant and look more realistic.

Databricks' Carly Taylor 3:32 You want more polygons? You want things to look more real? Like, you need a better GPU. And, you know, that that consumer drive forward not only, you know, made NVIDIA the company that it is today and allowed them to innovate and be the best in the world at doing what they do, But it drove the price at every point of those GPUs down. And you can actually see this trend over time when you look at basically the cost to do a calculation on a GPU. The best way to normalize it is, what does it cost to do some sort of matrix calculation?

Databricks' Carly Taylor 4:03 And the cost for each calculation over time has just gone down continuously. So now we're at the point where we have these insane data center GPUs that are used primarily for generative AI. They're not really used for gaming. Although, that would be sick.

Conor Bronsdon 4:20 Oh, I mean, I can only imagine what that You're right. Use all the capacity, but it should be fun. Running it, everything at, like, the highest setting possible.

Databricks' Carly Taylor 4:30 Your electric bill is, like, $80,000. You know, now we're we're at this point where it's it was possible to build those data center GPUs at the price, the point that they're at today. And that's not to say they're not expensive. I think one of those is, like, $70. Those aren't for people to buy. Those are enterprise level. But, you know, without the push, the consumer push for GPUs and the acceleration in that development and making things cheaper, making each of those calculations cheaper over the years,

Databricks' Carly Taylor 4:56 I don't think that we would have been primed for the AI revolution that we have today. Because the compute power necessary is just incredible.

Conor Bronsdon 5:03 Absolutely agreed. And it's interesting to see how the Kudo ecosystem and everything NVIDIA has built there really set the stage for this last ten plus years of development in AI. And as you mentioned, you've gone deep on this topic before, and maybe we'll have to have you back for another full episode where we just on that. But I love that you've been able to carve out such an interesting niche for yourself here as a director of engineering at Activision, then Microsoft, a founder,

Conor Bronsdon 5:28 an influential industry voice with over a 170,000 LinkedIn followers today. You've obviously done a ton of public speaking, and you're one of the lucky folks who gets to talk about video games and their impact on AI and data every day. What first made you fall in love with gaming? Was it roller coaster tycoon, something else?

Databricks' Carly Taylor 5:47 I think that was definitely part of it, if I'm being honest. That was one of my favorite games. But I I guess I've identified as a gamer ever since I was really young. I started playing video games with my mom when I was probably too young. Nowadays, they'd say, know, don't let your kids sit in front of screens, but I was, like, this far away from, like, an old CRT TV

Databricks' Carly Taylor 6:07 just with my nose straight up against the glass. You know? And I think that as an only child, like, experience of not just playing with someone, playing with my mom later, gaming on you know, like, sitting next to each other on the couch, playing video games with your friends was, like, such a just such a vibe. And then growing up into multiplayer online games,

Databricks' Carly Taylor 6:29 I think that it's just been something that has been part of, for me, like, recreationally, but part of my life since I was young. And then it really exempt like, I think the role gaming played in my life was really exemplified for me during COVID. Like everyone else, I was stuck inside. But also like everyone else, I was playing Warzone basically constantly with my friends, and it was a great way to talk to people because we were all so isolated.

Databricks' Carly Taylor 6:57 So we'd all sit down, make a drink, hop on, you know, and just kinda just, like, talk crap with one another and hang out. And, yeah, it was a really good time, and it was at that time that I saw the job posting at Call of Duty, and they were looking for a data scientist. And I remember talking to my squad on Warzone and saying, they're looking for a data scientist for Warzone. Wouldn't it be crazy if I got this job? And, like, the rest is history.

Databricks' Carly Taylor 7:25 Yeah. Love that. I think I've just been a gamer. I don't know anything different.

Conor Bronsdon 7:29 That definitely resonates with me as well. I'm right there with you with Roller Coaster Tycoon. And then for me, like, my first time really getting into coding, like, I'd I'd started to do a little bit, helping refurbish computers on, you know, basic stuff around getting, using a command line to, actually refurbish them and, you know, update the software. But then it was like building websites for a RuneScape clan I was running. I was like, honestly, for my first time doing, like, quite a bit of coding. So,

Conor Bronsdon 7:56 it's funny to see how gaming will drive folks to take on these new challenges. And, you know, you mentioned the social collaborative element. I was doing a ton of Among Us during the pandemic, where it's like my friends and I would get together and be like, let's do a little murder mystery together. So I I totally hear you on that. And it's it's really cool to see how you've not only been such a integral part of the industry and seen it up close,

Conor Bronsdon 8:19 but have applied your background as a as a coder and data scientist, and looked at uniquely, I think, how traditional game development has has a kind of a data problem. So let's set the stage for this conversation with some grounding in how game development happens, plus how game data is captured, since I think that'll help inform this whole conversation around applying AI to gaming and how it flows

Conor Bronsdon 8:47 into one another. So we'll dive into the intersection of AI and gaming a bit later, but can you walk us through how games are traditionally built, particularly focused on how data and telemetry fit into that process?

Databricks' Carly Taylor 8:59 For sure. Yeah. I think it it really it depends. Like, the traditional game development process is gonna vary based on if you're at a startup, if you're bootstrapping things yourself, if you're a triple a game studio, it might look different. I think the unifying theme, though, is that game development is creatively driven, which is good. Right? Like, that's how we get these immersive worlds and these really enriching stories.

Databricks' Carly Taylor 9:24 The issue with traditional game development though is that when you're coming at things purely from a creative standpoint and you're translating that into engineering requirements for how the world needs to be built, you're thinking about data, but you're thinking about data to accomplish a singular goal, which is to get the game out the door. Right? Everyone wants to make ship. Like,

Databricks' Carly Taylor 9:47 that's the goal. Right? And you see when things get pushed back, like Grand Theft Auto just got pushed back again, like, there's reaction, and there's a cost to not shipping your game. And so, like, these deadlines are taken very seriously. And working in gamedev, I can tell you that, like, you are constantly counting down the minutes until you go to alpha, until you go beta, and then until you go to ship.

Databricks' Carly Taylor 10:11 And, again, driving that process is this creative iterative process. There's green lights. There's all sorts of things that studios are balancing and juggling and trying to get just to get the game out the door. And from what I've noticed, oftentimes data ends up becoming a second class citizen in those conversations. Because while critical to the game development process and specifically to the live operations process, which is how you support your game once it's out,

Databricks' Carly Taylor 10:40 you can still ship a game without data. You just won't know anything about what's going on. You know, there's things in telemetry that you need to know. And as the game is being built, engineers put this in because they need to see if things are working properly. Right? They want to capture, did you or did you not kill that boss at the end of the level? Right? Like, those things are very critical to know. There's other things that might not seem

Databricks' Carly Taylor 11:04 important until you realize they're extremely important. Things like the session length of a player, if they got stuck in some onboarding step somewhere. Right? If your controls or your mechanisms or your game design has something that's too hard for people to pick up or something that's unintuitive. Those are things that you can answer with data, but you have to be extremely purposeful about how you're defining the scope of what you're trying to understand

Databricks' Carly Taylor 11:34 and how you're gonna reflect and look back upon the decisions that were made. And the unfortunate reality is that when you're coming up against ship, the first thing to slip is like, well, you know, whatever. We'll figure it out when we figure it out. Or we'll see it in game reviews if people can't get through onboarding. You know, we'll hear from the community.

Databricks' Carly Taylor 11:51 And there is some piece of that that's part of this. But the in game telemetry, again, often just ends up becoming a second class citizen because shipping is the most important thing. And it's never unless it's something extraordinarily important or some financial metric, it's never a block to ship a game. It's just a really nice, nice to have.

Conor Bronsdon 12:12 It's such an interesting, I guess, illuminating fact because, you know, from the outside looking in as someone who's just a casual game player not working directly in the industry, you know, obviously, the term AI has existed in gaming for decades and, you know, referring to NBC behavior. And so my expectation was just, oh, of course, there's so much data being collected because they have to fuel these,

Conor Bronsdon 12:35 you know, nontraditional actions, potentially deterministic potentially nondeterministic based off of, you know, how a player is interacting with the game and, you know, the settings you have applied. So it's it's almost confusing to think, oh, data isn't a first class citizen when it comes to gaming? So, I mean, you mentioned this pressure to ship as a big part of

Conor Bronsdon 12:56 why telemetry sometimes kind of gets forgotten here. Is part of the challenge here that studios aren't really having the conversation in advance about what do they actually want to leverage that data for, until after the game is live because they're just saying, hey, we gotta ship this thing? Or why do you think these kind of crucial data strategy conversations are happening

Conor Bronsdon 13:21 often too late in the process?

Databricks' Carly Taylor 13:23 Yeah. I think that that's a really good a good question, and that's a good guess. I think that there is a lot of data. It's not always an issue that the data isn't even being collected. So that is definitely part of it. If you're not actually instrumenting the telemetry, like I said, if it's a second class citizen and it's not there, it's not there. But the question then becomes, even if it is there,

Databricks' Carly Taylor 13:46 what are you doing with it? You know? The gaming industry is world class in a lot of things that they do, but I will say that the data adoption curve for most gaming studios is probably behind where you would consider most tech companies in terms of best practices for where the data needs to go. And a lot of that has to do with every game is so different, and a lot of them have these extremely complicated,

Databricks' Carly Taylor 14:11 especially for online games, like server client architectures. How often are you putting data where it's going? Are you saving all of your data? Are you sampling it? Where are you putting it? How are you transforming it? And then that's just the question of where does it go? Then you have all these other questions about how do you transform it and get it business ready?

Databricks' Carly Taylor 14:31 And those aren't always immediately obvious answers, especially since the people handling the data are game devs. They're not business analysts. Right? Like, they know what they care about in their game, but that might not align well with what finance or marketing cares about. Right? And in order to find a harmony where you have everything captured that everyone could possibly care about. You end up in a space where you're doing a lot of trial and error. And, unfortunately, in the game space, trial and error is expensive.

Databricks' Carly Taylor 15:01 Dev time and time to get things into the, you know, live build. Like, we're talking six, eight weeks sometimes. You know? So missing something is very expensive. And I think it's really just now becoming so obvious that gaming not only is a huge opportunity in terms of the number of people who play video games. I think we probably had this idea of the industry from twenty years ago, where the only people who play video games are like, you know, the kids in Stranger Things were also playing

Databricks' Carly Taylor 15:35 D and D. Like, it's like this weird nerd habit. And I don't think that that has proven true. I think mobile games have opened up a whole new world for people. And so when you have anything that's being massively adopted across the world, you're going to have behaviors that you don't understand. Even if you've played your game a thousand times, there it's nothing like having a thousand different people playing your game once. Right? Like, the

Databricks' Carly Taylor 16:07 possibilities, like, the feature space of things people can do, people break things very creatively. Or, like, you know, and that's just part of the human experience. And I think, you know, unless you've done it before like, these triple a studios are getting a lot better with data because they have to do it all the time. But it's still been in the past maybe five years that data's become something that's taken more seriously and that has a proven ROI.

Databricks' Carly Taylor 16:33 But if you're not set up to you might be even capturing your data. If you're not set up to act on it, then you have another problem. Like, are you prepared to do experimentation? Are you prepared to make changes if you see that people don't like some mechanic? Like, are you prepared to just, like, rip something that you like out of your game based on feedback?

Databricks' Carly Taylor 16:52 It sounds obvious, but some people are not willing to do that because they feel like they understand things in a way that data can't capture. So if you don't have a data first mindset of the people who are looking at it, it doesn't matter what it tells you or if you're capturing it or if you're seeing anything, because no one's gonna listen to it. You're bringing up so many good points here.

Conor Bronsdon 17:13 And I think where I wanna start is maybe the most basic example of this idea where we're always designing for how we think a customer is gonna interact. No matter the business, whether whether it's it's gaming or something else, we think, oh, they're gonna do this. And there's always a creative way that they break it, and they do it differently from us. And I I wanna talk about this difference between

Conor Bronsdon 17:36 QA ing and testing and games and game design versus how things happen in actuality as players actually get exposed to it. But to me, it almost is simpler. Like, there's this very common video, we've probably all seen it, of a cup with little holes for shapes in it. And it's like, oh, like, what what shape do you put in which hole? And every shape goes in the square hole. Because like, oh, they all fit in there. It's we don't need to put it in these design holes. And I always think of this image. People who have listened to me in different podcasts have probably heard me reference this before, because it's just the one that always sticks to my mind when we talk about this, like, design problem of how is a user actually going use this? It's like, oh, they're going to find the basic use case. They're going to apply it,

Conor Bronsdon 18:19 and they're going to go, oh, I can spam magic missile in this game. I'm just going to keep hitting people with magic missile. Don't need to try these eight other things, I can just hit them with more magic missile. And this comes back to that difference of of QA teams playing a game versus how the general public plays it, and like how a thousand weird gamers like myself are gonna go break things in different ways. How does this gap impact

Conor Bronsdon 18:39 the the usefulness of data collected before a game hits beta or a full launch compared to once you have, you know, all these different folks trying and breaking the game in unique ways?

Databricks' Carly Taylor 18:50 It's crazy because you don't realize your own blind spots or your assumptions until they're tested by people who have who are coming at it completely, like, with brand new eyes. Right? A lot of this can be circumvented if you have, like, a good user testing framework. Right? So a lot of the big studios will bring in random people literally off the street for

Databricks' Carly Taylor 19:13 playtests to see how they play. And I've seen some of these playtest areas and they're crazy like they do eye movement tracking across the screen and see how you interact with things. And that's very high-tech and can get you a lot of the way there. But it still cannot account for regional differences. Right? Like, are you doing play tests in every market you're in? Like, probably not. Like, you try to get good demographic data, but it's hard to. So, like, are your translations in game landing in some

Databricks' Carly Taylor 19:46 country where something is not a norm and people kind of don't understand what's supposed to be, like, a well understood social interaction that leads to some mechanic. Like, it's easy to take those things for granted. And it's also easy well, not easy. It's hard to truly test the resiliency of your infrastructure at scale. So companies will obviously do stress testing of their data pipelines, of their servers, of all of their architecture to make sure that they can handle what they would expect at, like, a launch capacity.

Databricks' Carly Taylor 20:21 You know, they'll stress test all their systems and make sure that they're not gonna completely implode. But there's nothing like a production environment to truly test the things that you've built. I posted this on LinkedIn the other day. It's it's like a funny joke, but it ends up being true in more industries than just gaming, where you have users saying like, hey, can we hire more QA,

Databricks' Carly Taylor 20:42 you know, to test the product before it goes out? And the company just says, we have QA. They're called users. Yes. And the sad truth is that it ends up being, like, a lot of weird mechanics, bugs, holes in the maps, like, you know, just stuff that is kind of hard to find doesn't get caught until you unleash thousands and thousands of people who are getting into crazy positions and taking vehicles where they were never meant to go, and just doing all the wild stuff that a bunch of roving bands of lunatics are gonna do. You know? But as you mentioned earlier, this also creates a secondary problem of

Conor Bronsdon 21:18 how much are you actually going to react to that user data. Mhmm. And it sounds like there's a lot of disparate approaches across the industry and maybe hurdles that have to be overcome to push an update and actually react quickly if something is buggy and broken.

Databricks' Carly Taylor 21:35 It's hard. Yeah. So, I mean, you're thinking, like, a lot of gamers don't realize this, but let's think about a game that is on PlayStation, Xbox, and PC. Right? Like, the standard stack of platforms. To get a game update, let's say, like, just on PC. You know? I worked in security. Let's say I wanted to do an update to a kernel driver that's only on the PC version of the game. So I don't have to touch PlayStation or Xbox, but I need to send a PC patch out. How often do you think the, you know, in this case Microsoft for PC updates, like, how often are they gonna let me make everyone

Databricks' Carly Taylor 22:18 who plays my game update just to get a security patch on? Like, can I make people download an update every day? What if I have to do it across all the platforms? You know? Like, how how much of an update is worth updating for? I'm gonna ask someone in rural Australia to pay, like, a crazy amount because they went over their data cap this month because I forgot to put a piece of telemetry in the game, so I have to ask them to download date.

Databricks' Carly Taylor 22:47 Like, there's certain things that just are not critical enough to justify the user friction to push out things like that. Not saying security. Security is a really big one that is justifiable, but there's other things that are, like, nice to haves that sorry. I'm not gonna make, you know, millions of people around the world download an update because you forgot to get a session length telemetry marker in the game. Like, it's just not gonna happen. So, like, you'll get that data when you get it, but it'll be maybe in a couple weeks. And that's if you can get, like, a fast patch out. For other things, I mean, we're talking like a build process.

Databricks' Carly Taylor 23:23 Six weeks usually is the time frame you're looking. Like, you're looking at your new content for a season update. Like, you're months ahead. You are testing early. These build pipelines are long because you have to make sure you're not gonna break things with PlayStation. You're not gonna break things with Xbox. Like, they have to all play nice together. It has to get into their store. Like, you know, this actual pipeline for development,

Databricks' Carly Taylor 23:47 we have to do our own internal QA. PlayStation has their own QA for certain things. Like, there's a lot that goes into making sure a patch is safe and is okay to be put into a game. And data's usually not one of those things that you can say like, hey, can we spin up overnight QA resources because I forgot this piece of data? Like, we're gonna need to pay people overtime to work overnight

Databricks' Carly Taylor 24:10 because we have to change this patch, and everything has to be QA tested before it goes out. Because you never know. You can put a piece of data in, and it could break something. I've broken games before with data update. Actually, one I did really badly because I'm I flipped a greater than, less than symbol. That was a fun one. But you the simple mistakes. Oh my gosh. Always. So you have to I mean, you have to test that stuff. Right?

Databricks' Carly Taylor 24:34 And so the pipeline for that is long. And if it's not an immediate thing that needs to be done because it's a breaking change, like, you gotta wait.

Conor Bronsdon 24:44 So from your perspective, what does it look like for a game studio to actually unlock the full potential of their data? How can they more effectively design their games to get the data they need from the start and then leverage that later?

Databricks' Carly Taylor 25:00 You know, I think it starts with really the question that everyone wants to know. Right? Not just in gaming, everywhere. How do I get value out of my data? Everyone has too much data, and everyone wants to know how do you get value out of it. Nobody wants to just ignore it and leave it on the table. Right? I think in the year 2025, very few people are like, no, I don't use data. I just go off vibes. You know? So if we start from the question of how do I get value out of my data if I'm a game studio?

Databricks' Carly Taylor 25:29 Well, you need to figure out what kinds of questions you're always asking yourself, what kinds of things that you feel like you have a good handle on, and what kinds of things you feel like you have no idea about. Like, when you last shipped something, what were you completely blindsided by? Was there something people loved or something people hated that you should have probably seen coming?

Databricks' Carly Taylor 25:48 Was there something that broke that you probably should have been prepared for? These kinds of things will come up the longer you do it. And the more you realize like, okay, the more postmortems we do, the more retrospectives we do, the more we look at what every executive is asking after every single time we launch, we'll realize, okay, these are the low hanging fruit questions that we're always trying to scramble

Databricks' Carly Taylor 26:11 to answer. And then you start incorporating data, people, whomever that is, that could be your data engineers who are handling getting the data somewhere, that could be your head of data analytics who's for, who's on the hook for actually answering the questions once the data comes through. It should be all of the above. And you should be including these people in the development process probably pre green light. Like, before every green light session, you should have at least one debrief about like, how are we looking with telemetry? Are we gonna have the things that we need to answer the questions that we deem are important?

Databricks' Carly Taylor 26:43 Like, what are we trying to accomplish with this game? Right? Or is it a sequel? Are we trying to get new people? Like, it might seem obvious, like, you want people to play it. Right? But I think every game probably also has some other things they're trying to do. Are they trying to further some IP? Are they trying to tell a new story? Are they trying to reach a new market? Are they trying to change from hyper casual games to more, like, you know, serious games? Like, what's the studio trying to accomplish with this game? And then what kinds of questions do you think we're gonna answer? You'll never get comprehensive, but if you start

Databricks' Carly Taylor 27:15 asking yourself that early, you won't find yourself caught flat footed. And I can tell you the worst time to be caught flat footed is often during your beta. Because for players now, it used to be that beta was like, well, if it's, you know, broken in beta, we'll fix it before we ship. Right? Like, beta is where stuff's supposed to be busted. But what you'll find is that you are asking the questions

Databricks' Carly Taylor 27:43 of your beta cohort that you're gonna ask in the live game. And that's great if you're like, oh crap, we can't actually answer that. That'll happen. But you shouldn't go in totally blind and say, well, we don't have any data. These are all the things we wanna know. We won't know because we didn't have it for the beta, so at least we'll know by the time we get to live.

Databricks' Carly Taylor 28:03 Because you can't act on anything you could have learned in the beta. All you're learning is that you weren't prepared from a data perspective. And then players also have an expectation of betas now, a bit more polished. It's gonna set the stage for how your pre orders are gonna look. It's gonna set the stage for what people's expectations are. It's gonna set the stage for reviews.

Databricks' Carly Taylor 28:24 And so if you are going into beta thinking beta is when we're gonna make our data decisions, like, you are way too late. Like, even alpha probably. Like, it's a good time to have those questions if if you've waited that long, but that might also be too late. Because depending on how much telemetry you're missing, you're touching a lot of pieces of code, and that's risky.

Databricks' Carly Taylor 28:47 You know? You really wanna be doing this as you're building so you can kinda derisk it and kind of build gradually as opposed to, like, a mad dash at the end.

Conor Bronsdon 28:57 Yeah. You're you're absolutely right that there is, I think, too often, not not just in gaming, but many industries, this mad dash at the end of, like, oh, shoot. We have to go fix this thing that we kind of knew we were gonna have to do, and we didn't plan for it necessarily. And as we've alluded to throughout this conversation, one of the key trends that is reshaping how we leverage data today in gaming and beyond is AI.

Conor Bronsdon 29:20 And while AI has existed in gaming for, as we've referenced, things like NPC behavior for for years, there's obviously a lot of new stuff here with generative AI. How does this traditional game AI contrast with the the new generative AI wave and the updates that have happened to machine learning as it's exploded over the last twenty years. How are you seeing

Conor Bronsdon 29:45 all these changes in game data and these improvements in telemetry intersect with the, I guess, the opportunity for gaming studios from new AI technology?

Databricks' Carly Taylor 29:56 You know what is so funny about this conversation about AI is that as a someone who was involved in both the industry side and now on the vendor side at Databricks, you know, one of the biggest companies in AI, I've seen I've personally witnessed these conversations happening where I'm kinda just a bystander. And the way that came studios and vendors are talking

Databricks' Carly Taylor 30:22 past past each other without even realizing when they talk about AI is, like, hilarious to me, but also, like, an easy area for me to make an impact at Databricks because I'm like, hey, guys. Here's where we're completely missing the mark when we talk to them about this. Like, I can tell you you are having three just completely parallel conversations, and no one is actually talking about the same thing. And you think you are, but that's why there's so much friction. Like no one's understanding each other. And that's because of exactly what you just said.

Databricks' Carly Taylor 30:53 Game studios, developers, gamers have been talking about AI forever. We were probably the first people. Not only did we set the stage for, you know, AI as we think of it today with GPUs and adoption there, we also set the stage for people to even use the word AI. I mean, maybe not, but, like, come on. We popularized it. We're the ones that used it colloquially the most. Right? Even calling someone an NPC now is like a meme. Right? Like, gamer culture is everywhere. And so

Databricks' Carly Taylor 31:23 when you have a vendor come to a game studio and say, like, we're gonna help you build intelligent AI, they're first of all thinking in game AI. That's all they're thinking about. They're like, okay. So you're gonna make our NPCs not so stupid. That's great. You know? That's feedback we get all the time. Like, yo. You know? You're we love the game, but, like, how could it be more immersive if so and so didn't have, like, a specific talk track? Like, make it a bit more free form.

Databricks' Carly Taylor 31:51 That kind of thing is good. But the issue becomes when you come to a game studio and you say, yeah, we're gonna make your AI, like, you know, do x, y, z. Even if we're talking about in game AI. We're gonna make it so smart. It's gonna be able to talk to the player. It's gonna be able to remember everything they did. It's gonna do blah blah blah blah blah blah. All these lofty promises.

Databricks' Carly Taylor 32:11 What no one, except for me, to toot my own horn, in the vendor space has been able to tell anyone is like, okay, well, how are you gonna do that? Like, you realize we're resource constrained. Where show me. Like, is the compute to do this in the room with us? Like, how? You know? Every in game AI has been, like, hyper optimized to run because that's what game companies that's what gamers have to have had to do going back to Roller Coaster Tycoon.

Databricks' Carly Taylor 32:40 You gotta make this actually work with what you got. Right? And not everyone has the $70,000 NVIDIA GPU ready to go to rock the latest game that you I wish I had that. Right? I know. Right? Like, we're dealing with min spec, like, the minimum specifications to play the game. So not only that, but the question of LLMs within gaming is super interesting to me because it's like, okay.

Databricks' Carly Taylor 33:06 In games, traditionally, you have two choices. You will run something on someone's client, so, like, on their PC, on their PlayStation, whatever it is will be locally computed, rendered, whatever. Or you'll run it on a server if you're in a client server architecture. That will be run-in some data center that everyone is connected to that server, and then whatever the server does will talk to the clients and tell them what it decided. Right? Those are your only two options for where to run things, usually. There are constraints on both of those, to your point. A 100%. Right? Like, normal NPCs run client side. Right? Like, they just have to, like, walk around the map and whatever. And some of them are running on the server, like, depending if it's a bot. But if you're playing an open world single player game, most of that's local.

Databricks' Carly Taylor 33:50 If you're like now we're introducing, let's say, okay, yeah, we'll just put an LLM on it, someone's proprietary LLM that we train to make the NPCs have cool conversations. Where are you going to do that inference? On the client? You could, I guess if someone had a really good GPU. But what are you going to send your trained model to someone's computer and just give them your IP?

Databricks' Carly Taylor 34:16 Model weights for LLMs are worth hundreds of millions of dollars, billions maybe. Think of OpenAI. Are they gonna give you their trained model weights for ChatGPT? That's their IP.

Conor Bronsdon 34:29 Like I guess they probably never do that. Realistically have to use, like, a local instance of, like, a LAMA model, for example, as, like, really the only real Like an open source something? Yeah. Like, and that's cool. So then how do you

Databricks' Carly Taylor 34:42 integrate into that? How do you make sure that it is gonna respect your IP? That it's gonna maintain the voice of the character that you like. Does that feel good? Okay. Well, let's say then you're gonna fine tune it on your own data. Well, now you have an IP issue again. Right? Because now you're leaking your IP back by giving it to people to run locally. Okay. Then you say, well, then run it on the server

Databricks' Carly Taylor 35:04 and just send the information down. Game surfers don't typically have GPUs. They don't need them because they're not rendering anything. They don't have screens. It's a server. So, like, where are you so the question becomes, you're telling me you can do this thing, but you're not actually giving me any ideas for how I can realistically do this. Now there's another option.

Databricks' Carly Taylor 35:28 It's not just the server or the client. You know, you can have something else in the loop. Like, we have this all the time. We have VMs that sit next to servers with the data. Right? Data comes out of the server, goes somewhere else. Like, those aren't the only two options. You can run things and send them down to the client that doesn't come from the server, or you can send information to the server to send to the client.

Databricks' Carly Taylor 35:48 That's all reasonable. But no one's really talking to game studios about that. They're just saying, oh, yeah. Just put an LLM on it.

Conor Bronsdon 35:55 Yeah. I think this speaks to the kind of common problem that's happening in a lot of, call it, AI marketing today of just considering LNLLM as a magic bullet instead of a really useful tool that has constraints, to your point. And it feels like the opportunity with gaming for the moment with AI beyond the MPC piece already, which has obviously been there for quite a while, is more about like, okay, how are we leveraging all the data we collect

Conor Bronsdon 36:25 and applying AI to our analytics? Or are there QA opportunities with LMs, and and throwing them at games earlier so that you can have that user experience of, like, all these disparate testing? I'm I'm curious if there's much happening in those directions so far.

Databricks' Carly Taylor 36:42 See those as being much easier problems to solve and much more realistic problems to solve. So specifically with AI with your data, right, the good thing about modern data infrastructure, Databricks in particular, but every good data provider and product suite is offering something similar, is that once you have your data in a place where it is centralized, it's governed well, so you're legally compliant,

Databricks' Carly Taylor 37:10 you have everything a place that needs to be, You have all of your reporting built on top of it. You're understanding your business cases. You're building your business ready, gold standard, gold layer of data. Principles handled. There are tons of products that are integrating directly into your data warehouse, directly into your data catalogs, like with Unity Catalog and Databricks, like, getting directly in there and you can get AI natively within the platform.

Databricks' Carly Taylor 37:42 So if you have good enough metadata, which is just a description of what all your data actually is, which is actually harder to come by than you would think in gaming. But once you have good metadata, you can ask AI to almost anything about your data. Right? Like self-service analytics is here in a big way. Like, in a way that even a year ago, I wouldn't have believed it would have been possible and it had accelerated this quickly.

Databricks' Carly Taylor 38:07 So, like, we're opening up doors for nontechnical people, people who don't write SQL every day, to ask questions of their data in plain language where they can say, like, hey. How did, I don't know. How did that skin sell in the in last week's, like, you know, 04/20 sale? Like, Google, we did a collaboration. You know? Did people like it? Or, you know, what types of skins are people loving?

Databricks' Carly Taylor 38:35 Or what was our most popular game mode in the last week? Did we see any crazy things pop up on Steam this weekend? You know? Have we seen any data outages? All of that stuff can be somewhat automated and somewhat self-service, which is crazy and incredible. So that's part of the AI that when you go to game studios and you're like, We're going help you build AI. They're thinking in game AI.

Databricks' Carly Taylor 39:01 They're not thinking, oh, I can have an exec self-service their small data questions so that my data team can continue answering the big ones and putting together the long form financial reporting, putting together these really intensive deep dives and experimentation, like a b test, causal inference things that we're doing. And they can just ask the silly question that they had about, like, do people like the new, like, you know, map?

Databricks' Carly Taylor 39:26 Like, the things that are, like, interesting but, you know, kind of derail conversations when you're talking about longer term analytics projects. And then the other one you brought up is very interesting, which is a bit touches on the NPC conversation kind of, but it also touches on machine learning and advances in machine learning, advances in in game bots that we've had going back probably ten years.

Databricks' Carly Taylor 39:50 And reinforcement learning is big here, and I'm talking about how do you help your human QA team by building more automated QA and reinforcing all the work that they do with automated QA bots. And that's also not an easy question to answer because you have to have a really tight pipeline. Because, basically, what you're doing is you're using your human QA testers

Databricks' Carly Taylor 40:20 to train the reinforcement learning bots, how they do their jobs. But then every time you introduce an update, the human testers have to take the lead because these bots don't understand these mechanics on their own. They learn from people. And they need to learn from QA because you don't wanna have them learn from players playtesting because now you're backwards. You're having Yeah. Players jumping

Databricks' Carly Taylor 40:45 The players are lunatics. Also, you don't want your users to be your testers. Right? You need to test this stuff before users ever see it. And so this ends up being a really interesting question and an interesting problem space. I've seen a lot of movement in the, like, automated QA. I don't even know what other people are calling it. Yeah. Just automated QA

Databricks' Carly Taylor 41:03 arena. I haven't seen anything yet that's, like, fully compelling to where I'd say, okay. I'm willing to, like, smoke test with a fleet of these bots and trust it. I definitely wouldn't trust it for a full QA pass. But I do think where it's gonna help is, like I said, if you have these crazy, you've gotta do an overnight test, and you have to just get good enough, and you need these bots to run through along with a couple of humans, and you don't have that many people online.

Databricks' Carly Taylor 41:34 I think that that would probably be the best use case scenario right now. Where as long as it's not fully breaking something, it has to go out because it's some sort of critical patch. But yeah, I think there's gonna be more advancements there as well. You just have to be mindful of like, anything, what does your what does your pipeline look like? Are there other

Conor Bronsdon 41:55 use cases or opportunities that you're seeing for AI and modern gaming today?

Databricks' Carly Taylor 42:01 Oh, tons. I'm building something right now with a group of people where we're pulling in Steam reviews. And not only are we, you know, leveraging that to try to do some of our localization, so, like, can we train an LLM or just use a prebuilt one? Can it understand the colloquialisms of gaming, and can it effectively translate different languages, understand sentiment of reviews?

Databricks' Carly Taylor 42:27 But also, can you then generate reports based on those reviews regardless of the language, but including localization information so that you know, like, oh, hey, there's something wrong with, like, the French version of the game, into, like, emailed reports for execs. Because that was something that we saw a lot was we had our community managers online basically twenty four seven

Databricks' Carly Taylor 42:51 having to scroll through Twitter, which, like, they should get a bonus just for having to spend time in there. And having to scroll through Reddit, which again Maybe a bonus again. Yeah. Yeah. Double bonus. And having to just, like, scroll through all these things and try to synthesize the information on their own. And the way a lot of that used to work is you'd have your community managers doing that. And then they'd say, okay, a lot of people are talking about,

Databricks' Carly Taylor 43:18 like, I don't know, the French version of, like, Bowser. Like, he sounds weird. And then they'd have to set up listening on their social listening tools for, like, French Bowser. And then they can go through the data that they're scraping and find instances of that. But someone has to know that they need to look for French Bowser to begin with. You end up in this chicken and egg scenario that you kind of have to know your problems in order to look for them in your massive data.

Databricks' Carly Taylor 43:43 And where LLMs succeed a lot here is that you don't have to give it any pre you don't have to say, Hey, go look for Bowser and tell me, like, where you see an issue here. You can just say, what are people talking about? What do they care about? Has there been any emergent conversations that seem different from what you're used to seeing? Are there any topics that seem like they're gaining traction?

Databricks' Carly Taylor 44:03 And then it can say, oh, yeah. People are mad about this French Bowser. Like, he does not sound right.

Conor Bronsdon 44:10 Honestly, I'm just struggling right now, because I I am in my head now picturing Bowser with a French accent, and I'm gonna have to go find this after the show. So

Databricks' Carly Taylor 44:18 I just made it up. But that would be hilarious. Ooey.

Conor Bronsdon 44:24 So I'm curious. As we're wrapping up here, do you see gaming continuing to push AI forward as it enables as it has enabled the GPU revolution or in other ways in the coming years? Maybe it's gonna simply add French and Spanish Bowsers, and we can experience those for fun. But, you know, what other ways is gaming gonna impact AI development going forward?

Databricks' Carly Taylor 44:47 I think, you know, in the ways that it continues to or has always historically. Like, gaming is really, for me, the the space that I think humans probably other than, like, writing. It's like the open world where humans can be as creative as they want. Right? Like, it's where you're the only limit to what you can accomplish is your imagination. And so as we want to be more and more imaginative

Databricks' Carly Taylor 45:12 with the worlds that we occupy, as we wanna do more and more with our technology, as we want more and more people to experience things simultaneously online, like, that all happens in games. Right? You think of, like, Fortnite, the events that they have. They've basically built best in class online networking systems. I've gone to concerts in Fortnite, which is weird to say. Yeah. No. But, like, when you think about the feats of engineering that it takes to put those concerts on, you realize, like, nothing would have ever propelled

Databricks' Carly Taylor 45:51 the needs of real time online networking and multiplayer experiences if it wasn't for gaming. Right? Like, who else would have been innovating in that space? Zoom? Like, maybe kind of, but they still can't even really handle big webinars. So, like, it's it's gamers who, like, they wanna do this stuff, and they're like, can we now? No. Can we figure it out? Hell yeah. So

Databricks' Carly Taylor 46:15 it's that spirit of just like, this is what I wanna build. This is what I wanna create. This is the kind of world I wanna see, and we're gonna do it. And it's so funny because I I see a lot of people talking about, you know, there is this inherent friction between creatives and engineers where they're like, wouldn't it be cool if we could, you know, whatever? And, like, my one of my favorite engineers I used to work with would say, like, yeah. If only it wasn't for the pesky speed of light. You know? Like Sure. Like, that's always the limit. Like, there's always some inherent friction.

Databricks' Carly Taylor 46:44 But the good thing is is that, like, the creatives I've worked with, like, their goals are so lofty and not grounded in things like reality or the speed of light or, like, you know, physics or where the rest of us live That, like, they force us to, like, constantly push forward. And while we might joke about it, like, there's so many things today we wouldn't have if it wasn't for gaming, and I think it's gonna be who knows where we're gonna end up? I think it's gonna be crazy, though. Is a great note to end on. And honestly, for anyone who wants to see where this all goes, I highly recommend following Carly on LinkedIn, where she shares so much interesting content and perspectives

Conor Bronsdon 47:20 on the industry and what's happening with AI. Carly, I am deeply appreciative that you joined me for this conversation today. It's been a ton of fun. Where else can our listeners go if they wanna learn more about you and about your work? You can find me, like,

Databricks' Carly Taylor 47:34 on LinkedIn, for sure. And I also write longer form content on Substack.

Conor Bronsdon 47:39 Fantastic. I'm gonna have to subscribe to that as well. And definitely check out all the incredible stuff that Carly is doing both with g g AI and Databricks. There's a ton of really cool AI and gaming things coming down the pipe, and we'll link everything, including Carly's LinkedIn and all these other links, in the show notes today. So Carly, it's been a distinct pleasure. Thank you so much for joining us. Thank you.

Conor Bronsdon 48:01 And to everyone who is listening, if you love video games as much as we do, maybe drop your favorite games in the comments. If you're watching on YouTube, we'd love to hear from you. Is there a game that got you excited about programming and technology? Or is this something where you're just kind of interested and you're following along? Are you a Stardew Valley person? Stellaris, that's me. Mario Kart, what playing? Are Is it are you Warzone here with Carly? French French Bowser.

Conor Bronsdon 48:27 Honestly, I may have to put French Bowser in like I'm gonna have to like sneak a French Bowser I'll in some find him with little beret. I mean, feels I will say if it's probably already a thing that I can just go look up. But if not, it's absolutely something I'm going to have to generate from So here's an AI use And while you're leaving that comment, make sure to like, subscribe, and follow Carly on her Substack and or her LinkedIn. Thank you so much for tuning in at Chain of Thought. We'll see you back here next week. And Carly, thank you again. It's been great. Thank you. Bye.