Data Federation, Not Centralization, Is What Enterprise AI Needs

Jitender Aswani 0:00 A traditional enterprise will have anywhere between 52 to 200 data sources. The federation is the only model that scales with entropy. Can centralization scale with entropy? No, because the entropy is constantly, constantly increasing. Imagine agents without access to all the data, without access to the right contacts. What would they do?

Conor Bronsdon 0:25 Every enterprise AI project that's failing is fighting the same hidden battle. Your agents can query the model just fine, but they can't reach the data, or they're not getting the right data. And the reason is often decades of architectural debt. We've talked about context and memory on this show, and we're going to dive much deeper today. Welcome back to Chain of Thought, everyone. I am your host, Connor Bronstein. Reminder to make sure that you've subscribed and turned on notifications if you're on YouTube. My guest today has spent 20 years at some of the world's most data-intensive companies before joining the team commercializing the open-source query engine he was customer zero for, which we'll definitely talk about. His argument sounds a bit like this. Data federation, not centralization, is the piece that enterprise AI has been missing. And with agents now generating a significant share of data infrastructure traffic and more than half of internet traffic, getting this wrong is getting more expensive by the quarter. That guest is Jitendra Swani, SVP of Engineering and Security at Starburst, the 3.35 billion enterprise data platform built on TreeNow. Before Starburst, Jitender led data infrastructure at Meta, Netflix, and holds multiple patents in machine learning, data management, and graph optimization. Jitender, welcome to Chain of Thought. How are things going?

Jitender Aswani 1:49 First and foremost, thanks for having me. I'm super excited to be here. And things are, things cannot be going any better for, for, for the industry and for Starburst. And I think you, you framed the challenge pretty well for the industry around data fragmentation. And now you talked about the growing volume of traffic from agents, and that is now posing a very interesting challenge for the industry. So I look forward to this conversation.

Conor Bronsdon 2:18 I do as well. I think your background is perfect for it. So let's start there. You were customer zero at Meta for Presto, which became what's now known as Trino. And it has become the foundation for Starburst as well for this over $3 billion company. What made you fall in love with this technology that has been such a through line in your career?

Jitender Aswani 2:40 That's a fantastic question to get started with.

Jitender Aswani 2:45 Before I talk about Presto, which eventually became Trino, let me take us back and I will age myself with that narrative as well. Before this technology, the world was starting to think about big data, and the technology that was dominating the big data landscape was Hadoop and Hive. As you can imagine, Meta at that point had roughly one billion daily active users, and those daily active users across the family of apps were generating massive amounts of data. And that data need to be analyzed to basically understand which users are active and what are they active in, which users go on to become monthly active users, which users are doing things in feed and where are they watching the video, where are they pausing the video. So a lot of this analysis was very much required for us to continue to improve the product experience. for 1 billion daily active users. And that was not very pleasant with Hive. Our engineers will write certain data pipelines and the queries were never interactive during those days. So the pipelines will run for hours. We will wait for data to get transformed and then data will eventually show up and product managers and engineers will look at how their product performed. And that was very slow, as you can imagine. It actually slowed the product innovation velocity. Meta is known for experimentation. And experimentation required quick thinking, quick speed, and course correction if an experiment is not going well. And you can't wait for 24 hours, 48 hours. Meta is known to make changes in minutes. So if your queries on this massive amounts of data are not coming back in minutes, you are starting to now impact the user experience for billions of users. So that was the world we lived in, and it was okay. We were totally fine. And then one fine day, the data infrastructure team starts incubating presto. And fortunately, my team was customer zero. We were in growth data engineering. As you can imagine, growth is all about growth hacking. I think the growth hacking term was popularized back in early 2010s. Presto comes along, and then you fire a query, and within minutes, the query comes back. Now, I don't need a break anymore. I don't have to switch context and go work on something else and come back, hey, has my query successfully run? But now with Presto, I'm just spending a little bit of a time looking over at another window and seeing my code, and then I come back and it's like, oh, I got this analysis back. Now this this A-B experiment had a pretty large blast radius and it's not doing really well. Let's cut this down. So that exactly what it meant is that the speed of innovation at Meta compounded with the advent of Presto and it actually I think it started a new era at Meta because product managers could do a lot more experiments. And they could very quickly course correct those experiments if experiments were not delivering the desired result. So I would say Presto was magical for us. So that's how it changed my life.

Conor Bronsdon 6:48 This is super interesting. Tell me more about this idea of Presto helping create a new era, because I think we often talk about technologies that are very well known as far as like, oh, AI is creating a new era. But more in general terms, we don't necessarily talk about the specific details that enable these new layers of instrumentation and experimentation.

Jitender Aswani 7:12 Yeah, so it unblocked many product managers and engineers who are working on improving the user experience in their product, or they launch a new feature, they would like to know how is this feature performing, right? Before I scale that feature up to go from 10 million users to 100 million users, let me collect some behavioral data. And that's a large amount of data. So to two benefits of it, Presto allowed us to query large amounts of data, but also the speed at which we can get the answers back was unseen before Presto's era. So as I said, it's unlocked a new generation of experiments for us. It just dramatically increased the innovation velocity at Meta. And I think that's why it became such a dominant compute engine in the industry. Pretty much every Silicon Valley company has it in their stack, whether it's Stripe or Apple or Net. I was at Netflix or LinkedIn. Pretty much every company picked it up because it just changed how many of these internet era companies started to experiment and started to take new innovation to the market. Imagine taking new product to the market and not getting any signals for days. because now you're collecting data, you're kind of transforming that data to make it queryable. And when you eventually query that data, the query itself takes hours, right? It discourages everyone. And three days later, the behavior has already shifted. Maybe your users have already moved on because you had certain feature in your product that did not do well. And we have seen examples of that in the industry, most recently with one of the, One of the smart speakers company, we saw that because the user feedback is super critical to understand how are your users adopting the product? Are they rejecting it? Are they loving it? And I think, I must say that Facebook's motto was ship love. This is how we used to say internally. We want to ship love. And we could only get signals if we could interactively query all of this data and see are we literally shipping love are we starting to piss off our users? [9:36] Conor Bronsdon: [OVERLAP] That is a lovely way to think about it. And I find it really fascinating that you were at ground zero of this, I'll call it data transformation. And [9:49] Jitender Aswani: [OVERLAP] yes

Conor Bronsdon 9:50 when you then took Trino as well, I guess, transition to calling it to Netflix, You were part of processing trillions of data points for, what, 300 million plus subscribers at the time, with heavy security components on top. What changed as you, you know, first did this at Netflix and then now started to take this same technology, or sorry, first did this at Meta and then started to take the same technology to Netflix?

Jitender Aswani 10:20 Yeah, the word changed from Meta to Netflix is. Presto started to mature. And as you can imagine, both these companies operate at massive scales. They're generating trillions of events because there are millions of subscribers for Netflix that are generating trillions of events, billions of users at Meta generating trillions of events. So now the technology also has to scale with the data volumes that are growing in exponential numbers. So that's the biggest leap we saw is that there comes a technology. Not only it's really, really, really, really fast and it's starting to unlock a new wave of innovation, but it can now operate at scale as well. So at Netflix, the number of use cases just dramatically increased for us. The other thing that I saw at Netflix was also the machine learning use cases started to run on Presto. And if all of us have seen how Netflix has evolved over time, you may have seen that your Netflix screen is very different from my Netflix homepage. That is because it's highly personalized. like personalizing Netflix experience. And this is not just one time personalization. Every time you log in, your screen is different. And that requires massive amounts of processing of behavioral data. Is that what has changed in the last 30 minutes since you logged in? Let's make sure that now when you log in, we give you an immersive experience so that now you are able to engage more freely, you're able to move around the Netflix screens, you're able to search better, and you can find your content better so that from search to what they used to say is the play, that time should be compressed as low as possible. That exactly, what enabled this behavior at Netflix is because of our ability to understand you, personalize it, so that your experience is as delightful as possible. And I think there is a business implication to it as not only we're delivering delightful experience, but we're continuously increasing the competitive mode. I think that's where the technologies, if you think about technologies, The reason technologies get adopted or rejected is, okay, what are the business outcomes? And one of the business outcomes always is that our competitive mode. Like Netflix has that competitive mode because the experience in Netflix application was delightful. And it all happened because of the massive investments they've made in the data infrastructure. and the ability to generate trillions of events that, okay, I could do that, but the ability to collect all of those events and then process those events and extract out signals that are relevant to each and every individual. And we're not looping people into cohorts and say, this cohort of people coming from this zip code should be given this. We're personalizing at individual levels. So imagine doing that for 300 million subscribers every day, multiple times a day, we're processing that much data. and delivering delightful experience to our subscribers. That's the competitive mode. That's why people in the early days loved the Netflix experience. It was so engaging, so hooking, that there was nothing else like this. I think that's where a new generation of personalization applications started, both at Meta and then at Netflix.

Conor Bronsdon 14:07 I love that you make this connection to the user value that gets delivered and what that means for the business. Because I think often with backend features, it's easy to not necessarily make that connection. So, and that speaks to the quality of the engineering work you're doing and the leadership work you're doing, that you are making that connection and saying, okay, here's the value we're delivering because we lowered the speed of queries in these batch modes we're running. Here's what's happening with the data architecture and how it matters to users and how it matters to the business. And that can be a challenge for a lot of engineering teams when they work on something that is considered by the business side, maybe more esoteric. So as you have this experience, you're customer zero at Meta, you're part of the early development, then you're at Netflix scaling out again, running engineering at a company that really commercialize a lot of this. What then brings you to Starburst years later?

Jitender Aswani 15:11 It's a fantastic question. Presto has been with me for almost 10 plus years now, and it has transformed my own professional experience. It has actually changed my own career trajectory. As you said really well, that the ability for the engineering leadership to be able to connect the investment they're making in the infrastructure to business outcomes. It should not be just, it's a back office investments. We should be able to articulate that the fact that we're making these investments allows us to accelerate business and deliver more and more value to our end users. Now, coming back to Starburst, I always differentiate between technology and a product. It's an amazing technology. And it has done wonders at many Silicon Valley companies. Now, can it be taken beyond a few highly tech forward Silicon Valley companies and can it be commercialized? So I think that the next challenge for Starburst was that we have this amazing technology has delivered incredible impact. Can it be commercialized? Can we take it to 10,000 plus enterprises? That was, that was, that has been the challenge for Starburst. And I think Starburst has, has, has, has lived up to, to that challenge. The fact that, um, 13 of the top 15, uh, financial services institutions in the world have, uh, very large deployments of, of, uh, of, of Trino and Starburst speak to the, the, the potential it has lived up to. And, and also there are so many other sectors where our customers just love Starburst and Trino. And that was for me was exactly that, okay, we have this compute engine and building a data platform around compute engine is quite challenging. The companies do not just need a compute engine. The compute engine has to work with the governance that is very much required at every enterprise. It also needs to work with various different other data platform services like ingestion. You're moving data over some streaming infrastructure. It lands in an object store in AWS or GCP or any other cloud service provider. Can a compute engine work in that ecosystem? In a closed ecosystem like Meta and Netflix, yes, you can highly optimize it. But then how do you take all of that optimization, package it? and offer it as a cloud service or offer it as a self-service option to your enterprises. So for us, that was the biggest challenge. Can we do that? Can we take our compute engine and build a data platform around it? And I think that's where Starburst really shines. That's where the mode is that it's a complete data platform. And the other big challenge was that Meta and Netflix had data fragmentation, but because these are the companies that are born and scaled in the internet era, they did not have that massive amount of data fragmentation that we have seen in traditional enterprises. A traditional enterprise will have anywhere between 52 to 200 data sources. And it's a natural evolution because their tech stack has been growing And new services are generating new data and it require a new type of data source like an object store or a NoSQL store. And traditionally they have been using relational stores like Postgres, Oracle, MySQL, and now you have MongoDB and Couchbase. And then suddenly you have S3. So you have this massive data fragmentation. And that was another challenge is that, okay, now we need to put our compute engine on top of this very fragmented data landscape, but still deliver that magical experience that I first had at Meta when I first ran my query on Presto. That's the challenge that we took on. we have very successfully delivered that challenge. And we continue to say that the data fragmentation is something we cannot avoid, right? We have to live with it. And the best way to live with that challenge is to actually put a virtual compute layer on top through data federation.

Conor Bronsdon 19:54 And I know beyond all the work you've done on the product side, all the work you've done on the engineering side, you have also done work on the academic side. You hold multiple patents. You've been part of research on this topic, on data management, graph optimization, machine learning. Where does this research instinct come from and how does it show up in the way you lead engineering organizations and the way you've helped develop Brasto and Trina?

Jitender Aswani 20:24 That's a great question. My research always starts with the interface itself. So as I was talking about the personalization use case, One of the research we did is related to understanding how will users in future interact with data. Back in 2011-2012, we felt that there is no Google for data. There's Google for information. You could Google structured documents, and these documents are unstructured documents. These are indexed really well. So you can just type in a natural language query, and you can get pretty incredible response. But that doesn't exist for structured data. You still have to learn. SQL is a very easy dialect to learn. And SQL has been around for 40, 50 plus years and it's not going anywhere. But many people do not know how to interact with data. And if you remember in 2010, 2011, when this whole big data movement started, there was this term coin called data democratization. Data democratization really truly mean is that now we have so much data, allow everyone to interact with that data. And curious minds will find some very noble solutions to the persistent set of problems that we have. And back in those days, we saw many such success stories that when you have so many curious minds working on on solving very hard to solve problems. If you give them access to data, if you break down all the barriers, any hurdles that exist, whether it's access to data, whether it's technology or like, okay, how do I write a query or how do I understand the data shape? That was the reason for our research. We felt back in the day that if I were to bring SQL search and graph together, And I'll explain why graph, but if I can bring SQL search and graph together, that you use a Google-like search box and you can interact with your data in natural language. And I'm talking 2020, 2011. Back then, the domain of natural language was very limited. It was primarily focused on natural language processing, NLP. And NLP went on to evolve, become natural language understanding, which went on to become large language models and transformers, and then large language models. And large language models gave us chat GPT and entropic cloud-like interfaces. So non-natural language conversations are relatively straightforward. 16 years ago, it was amazingly difficult. So my research was focused on introducing a new interface for business users so they don't have to understand SQL. They also do not have to understand where the data is and what is the shape of their data. Because in order for you to question, you need to know what data exists and how do I actually understand the shape of the data? And so there are these barriers and that's why you had these specialist roles of data scientist and data analyst, because that was their responsibilities to understand what data do we have, what is the shape of the data. how far this data goes back, what are the gaps in our data? A business user does not care, but if you give a business user as a natural language interface and say, okay, you can query your data because you have this very well curated data set that can answer your business questions. So just like the speed of innovation at Meta rapidly accelerated once we put Presto, we had a very similar that I would like all our business users to get access to their data. So they're not waiting for seven days or five days. So that was the premise for our research. And that required blending search, which there are certain technologies like building Lucene indexes on top of structured data, understanding metadata from the structured data. And then eventually, bringing everything together as an enterprise knowledge graph. Google search was based on this premise of a knowledge graph, but enterprises do not have a knowledge graph. Even till today, very few enterprises have invested in building an enterprise knowledge graph. That was the premise for our research. And I think, as I mentioned to you, when I approach research with my colleagues, we always think in terms of what will be the impact on end users? Will we be delivering a new way of interacting with the data? Will that change outcomes? So that's how I have always approached research, and I've been fortunate to get some patents on my research. And one more thing on this research topic is when we come across some gaps in our product. And then we see that our users are experiencing some friction in our product. I'll take a concrete example. Let's say you're searching for an answer. You have a question related to how do I, and I will use a very trivial example, how do I troubleshoot my VPN? And it's a very common problem. Sometimes people are having difficulties with VPN and they use the internal portal and they look for a response because they don't know where to go. Like, should I go talk to IT or put a ticket because I have a VPN issue? Or can I just search? Today, you can search. If you have Glean, you can search. And if Glean has indexed knowledge articles, it will be immediately able to give you instructions that this is how you troubleshoot your VPN. Imagine that problem 15 years ago. And then the cross collaboration, it basically means is that, okay, if there is no knowledge article related to troubleshooting a bad VPN connection, can I generate that? And this generation is not, again, this is all pre-generative AI era. Like, can I create a knowledge article by looking at knowledge article from internet? And I think that's where the whole research comes in is that I may not know the answer, but can I actually bring an answer and can I present a credible answer to you by, it's called, the most elementary term is collaborative filtering. It's like, can I actually look at companies that are similar to you and give you a response. This is how they solve their VPN problem. Or you also have a VPN from this company. They also have a VPN technology from this company. Let me just look up the FAQ from this VPN provider. That's the focus for our research is always be thinking about delivering impact to our end users.

Conor Bronsdon 28:01 You brought up several interesting concepts here, but the one through line for through it all that I'm seeing is this idea of data fragmentation and the fact that despite billions of dollars of spend, multiple different attempts to solve it, most businesses still have challenges here, whether it's using ETL, data lakes, lake houses, data stream, there's a billion terms for this. [28:26] Jitender Aswani: [OVERLAP] Yeah. [28:27] Conor Bronsdon: [OVERLAP] Uh, why is it that we're still struggling so much with data? Is it just related to the continuing increase of what we instrument? Is it because we now have this new era of agents where suddenly we have even more data of a different types that we're acquiring? I mean, we've been talking about this concept of big data for decades now and how we can leverage it. And we, we have [28:48] Jitender Aswani: [OVERLAP] Yes.

Conor Bronsdon 28:49 LLMs to help us with a lot of the data analysis now. Like, why are we still struggling here?

Jitender Aswani 28:54 It's a fantastic question. And I think you touched on two points, which have the response as well. Centralization was a fine, fine strategy. when the data was growing not as fast as your ability to move it. What has happened over the last 10 plus years is the explosion in number of SaaS tools, number of cloud services, and there are so many other vendors and pretty much every single service that you use is generating data and storing that data. So, up until a point, centralization works fine. But as the data volume continues to explode, the ETL tech debt starts to really accumulate. And your engineering teams are now struggling to think about, okay, how do I move? all of this data. And as you can imagine, data is not static. The shape of the data is constantly changing. Your SaaS vendors or your internal applications are now generating newer data. They may have added a new column. And now you have to go back and retool your ETL to bring that data in. Before that retooling, you're spending days debugging why my pipelines are failing. So if you are leading a data organization, you are constantly facing these challenges. You're having sleepless nights. In fact, I was talking to some colleagues at Meta and they said they were battling with certain, they call them SEVs, which are severity incidents. And like, what happened? Oh, pipelines were not running. So this is a constant problem. Industry and enterprises have spent billions building data infrastructure to move data from one place to another place. And then they keep doing it. And this whole data centralization concept surfaced during the big data era. Like move all your data to a data lake and all your problems will be solved. That's not true, right? Your problem starts to compound when you have moved all your data. Now you have a governance problems. you have this continuous ETL challenges. And there are so many other problems when you have centralized your data. So I think this, as I said, you had two points that have already answered the question. I think the federation is the only model that scales with entropy. Can centralization scale with entropy? No, because the entropy is constantly, constantly increasing. So as, as, as a, as a CIO, I'm always asking this question is that should I continue to have investments in having a single place, a single data gravity for all my data, or shall I have multiple data gravities? And I think the, what we have seen is, is a pretty strong pivot in that direction where federation alongside centralization is getting serious attention. [32:26] Conor Bronsdon: [OVERLAP] How has the new era of AI agents impacted the decisions you're making here? Because as we said at the top of the show, more than half of internet traffic is now agents and we're seeing them spread across every business. Many of us are exclusively using them for coding and [32:46] Jitender Aswani: [OVERLAP] Yes. [32:46] Conor Bronsdon: [OVERLAP] very rarely touching the code ourselves. There's so [32:49] Jitender Aswani: [OVERLAP] That's [32:49] Conor Bronsdon: [OVERLAP] many examples [32:49] Jitender Aswani: [OVERLAP] true. [32:49] Conor Bronsdon: [OVERLAP] here.

Jitender Aswani 32:50 Yeah. Um, in fact, 80% of our code is now generated by agents and our own, uh, uh, agent take, uh, invoices are exponentially increasing month over month. So we have seen an explosion of, of, uh, agents touching code base and agents requiring access to the data. I want to use this story that I've lived myself. So this one fine day, one of our senior VP walks out and asks us a question. Why is the churn in enterprise segment rising? And we all think we know, but we don't want to give him the answer right away. So three days later, after very thorough and in-depth analysis, we looked at every possible data source. We looked at every possible source of information, including emails and talk to AEs. We come back and we gave a response and at that point it's like, I don't know what you're telling me. Because you asked us a question and now we're giving you an answer why the churn in the enterprise segment has increased. I said, oh, I made a decision and I actually kicked off certain campaigns to reduce the churn in enterprise segments. But I said you did not wait for any response to the data question you had. It was like, oh, we folks are three days late. So now that's humans. And I have a similar story from another organization where the CFO asked the question, why is our churn up? And seven days later, three teams give him three different answers. Finance team gave him an answer and finance team had a different methodology. because they included cancellations. Product team has a very different methodology because product team included all the trialers, which was not included by anybody else. And then the sales team, which had the lowest share number, did not include renewals. So three different answers. And now imagine that this is pre-agentic AI era. we are struggling to get access to the data. We're all making decisions based on the data that I have access to. And not just that, there is this context problem as well, which we know is the hottest topping in the industry right now. And context is just not metadata, context is more than metadata. So going back to the discussion is that, that is pre AI era. Humans could not do it. Imagine, imagine agents without access to all the data, without access to the right. contacts, what would they do? And what has happened is that as a result of that, because these agents have this, they operate in this react mode, they reason, they act, they reason, they act, they are generating queries at insane speed. Some people call it at machine speed. I think they're generating it faster than the machine itself. And they're fighting multiple queries. And sometimes they're fighting unbounded queries as well. Because they're not making judgment of, shall I fire that query? They think they have access to this data, I'm gonna fire a query and see what I can glean out of it. I don't think I have an answer yet. They fired another query. And they continuously do that. As a result, we have seen an explosion in query volumes. In fact, when we launched our own MCP server, we saw the query volume go through roof. The MCP adoption through cloud for Starburst just exploded. As a result, many of our customers reached out because they had to scale the compute capacity behind it. So organizations have to be ready. They need this auto-scaling compute infrastructure, and they also need a very robust platform that can handle this query volume without compromising the query performance. So we have seen massive explosion, and I don't think we're going to see any

Jitender Aswani 37:21 slowdown in [37:23] Conor Bronsdon: [OVERLAP] No. [37:23] Jitender Aswani: [OVERLAP] the agentic queries that are coming into our compute infrastructure. [37:27] Conor Bronsdon: [OVERLAP] No, completely agreed. Like, we are seeing more and more desire for compute. I think it's exactly the opposite. I think we're going to see an increasing number of indented queries, even from where we're at today. And if I was a guess on a trend, I'd say it's more about what models are powering those queries. And it's going to be more about, okay, more open source AI that's cheaper, trying to optimize for costs and efficiency. But I mean, we're not seeing any slowdown in any [37:57] Jitender Aswani: [OVERLAP] Yeah. [37:57] Conor Bronsdon: [OVERLAP] of the areas that Everyone's sold out of their hardware. [38:01] Jitender Aswani: [OVERLAP] Yes. [38:01] Conor Bronsdon: [OVERLAP] It's pretty clear. I mean, like, look at the SpaceX deals with with Andropic, for example, [38:06] Jitender Aswani: [OVERLAP] With Entropic [38:07] Conor Bronsdon: [OVERLAP] where, [38:07] Jitender Aswani: [OVERLAP] and [38:08] Conor Bronsdon: [OVERLAP] yeah, they desperately [38:08] Jitender Aswani: [OVERLAP] Google, [38:08] Conor Bronsdon: [OVERLAP] need more compute. [38:09] Jitender Aswani: [OVERLAP] yes. Correct. Everybody needs more compute. I'm surprised Google is buying capacity from SpaceX. [38:18] Conor Bronsdon: [OVERLAP] I thought [38:18] Jitender Aswani: [OVERLAP] So [38:19] Conor Bronsdon: [OVERLAP] that was [38:19] Jitender Aswani: [OVERLAP] it's [38:19] Conor Bronsdon: [OVERLAP] fascinating. [38:19] Jitender Aswani: [OVERLAP] incredible. [38:20] Conor Bronsdon: [OVERLAP] Yeah. [38:21] Jitender Aswani: [OVERLAP] Yes, yes. And I think there is a small sliver of hope is that if agents get more guardrails. if they get more precise context. So they don't have to be endlessly firing these queries. I think that's where I feel there's a small hope because I feel that a lot of these agents are operating with a very light context. And the other challenge I see is that these agents do not have access to the entire data landscape. They have been isolated. So I think there are two challenges. Because of the data fragmentation and I would use my own example. the one of the FinOps agent that we developed internally on top of Starburst platform. When we were developing this data, we only had access to understand our own cloud spend. We had access to only data that comes from AWS. Now we do have data from Azure and GCS, but there is that problem. I was explaining earlier is that I need to now build the pipe to bring it over because it's not queryable. So we start, we build the agent on top of AWS data, but it doesn't give us full picture. So when I ask, here's our cloud spend, can you break it down by cloud service provider? And I know that it doesn't have access to other cloud service providers. But then it starts fighting queries. It's just frantically fighting queries without stepping back and thinking, do you even have GCS and Azure data? I think that's what I feel is that because it doesn't have access to that data, and it has that, I call some of these large language models which power these agents as like overzealous. They want to prove themselves right. They do not want to come back and say like, oh, I was wrong. I could not solve this problem. They have to always solve the problem. So they keep doing this. And sometimes I've seen that like after 30 minutes, they're still doing it. And I know you are wasting my tokens. Just stop. I sent you on a wild goose chase and you did go on that wild goose chase. So I think there's a small hope that with access to the right data and with complete context, maybe we will see that agents are now starting to limit the number of queries they fire. I'm not saying they will fire fewer queries, I think they will become smarter and they will structure their queries better and they will kind of also sequence their queries better. Right now I feel like they are kind of, sometimes they're stuck in an endless loop

Conor Bronsdon 41:10 Yeah, it's interesting to see how this changes over each generation of models and depending on the engineering of the harness you're using and what context specifically the agent has been enabled with. So I'd love to understand a bit more about how Starburst is building tools and data structures for agent context. I know you have an AI gateway product specifically for agentic workloads. I know you've had to make decisions around, you know, how to approach this. What's the approach at Starburst around development for agents?

Jitender Aswani 41:44 It's a fantastic question. So we're approaching it from many different directions because we know that there's not a single solution that is needed to completely harness the value these agents and underlying foundation models would provide. So first and foremost, we launched a new product called Ada, which if I were to go back and say, in 2011-2012, we were thinking about a search box. which allows users to now fire natural language queries on top of structured data and do not have to be concerned about understanding the shape of the data or understanding where the data is coming from, understanding do they have access or not. I think it's like, sometimes it does take 16 years to see that vision or dream come true. Technology was not right, but now with Ada, conversation analytics has become a reality. In fact, we're not the only company that are thinking about conversation analytics, because we believe the way traditional BI has been done is completely broken. I gave you a few examples of BIs that if I have a question, I fired a question, or I struggled to find which dashboard can answer this question, or which report will have question on enterprise segment is experiencing higher churn. But now with this conversational analytics interface from Starburst Call Data, I can just start asking questions. I think that's to us is a game changing product. And now if I were to peel the layers of this product, you mentioned something about the governance and the guardrails. So to build this conversation analytics, and as you can imagine, When I'm sitting inside cloud and I'm looking at a business problem, I've given it certain context. Once I have a concrete output, what do I do with that? I actually end up sharing it. I end up triggering a workflow out of that. That's how we have thought about conversation analytics. It's not just about ad hoc exploration. It's about turning that ad hoc exploration into some kind of a business workflow. So that's what we have introduced in Ada. It allows users to explore what data they have, prosecute data, and turn those insights into actions. And all of that can be done by sitting inside Ada. Just like an agent requires skills to grow its ability, Ada has skills. You can bring your own skills. let's say you're operating in a marketing domain and you want to perform a cohort analysis, you can encapsulate that knowledge around cohort analysis in a skill and ask agent, is that when you are interacting with this data, answer my question using these skills. So skill has become the harness for agents and agents have been able to Inside ADA agents are able to use the skills. Also, as I mentioned that turning insights into actions requires building a workflow. Inside ADA, we have the ability to talk to any MCP server. I'll use an example. I'm sitting inside ADA and I'm working on our telemetry data that comes out of our Cloud SaaS. I'm also now within ADA connected to Datadog using Datadog MCP server. And I'm understanding, I'm doing root cause analysis of an incident sitting inside ADA. Like this is the telemetry coming in. This is the query volume, as you mentioned earlier about agent query volume. We do see some hiccups sometimes. And I can sit inside ADA and review all the telemetry and say, there is this hiccup. What happened to the query performance? And I think that that was not initially possible because doing this kind of analysis will require us to build a custom application. Now with a conversational interface like this and with the ability to talk to any other service using MCP interface and drop any skill, now I have this super application that I'm able to use to supercharge my own work. And then coming to the challenge of context, in order for Ada to make accurate, decisions, it needs right context. So we have invested in building our own context layer. And context layer is not just a catalog. Catalog is all about like, where is my data? Who has access to that data? And

Jitender Aswani 46:56 what data do I have? So it basically answers three questions. But context is more than just like, the data and access to the data. Context is also about the ontology, is understanding that customer means account, it also could be, it also synonyms with opportunity, it also synonyms with this across my landscape, or the definition of churn, or the definition of cohort. Cohort has different meaning for different domains. And within a domain, cohort may have a different meaning for different individuals. So bringing all of this context together, all of that knowledge that has existed in individual's mind or has been somewhere buried away in Confluence or Google Docs, bringing all of that information together and turning that into a context graph, which is different from a knowledge graph. Knowledge graph is all about entities and relationships. Now context graph is building on top of a knowledge graph. These are the entities, these are the relationships. And here's more information about the entities. Here are business rules. One example of a business rule is for an enterprise, fiscal year may start in February and may end in January. But for some company fiscal year starts in July and for some companies there is no fiscal year and fiscal year maps to calendar year. So subtle terminology differences like this need to be made available to agent for them to make a smarter decision. The definition of churn, definition of ARR, definition of monthly recurring revenue, or definition of hundreds of metrics an enterprise uses, metrics around active users or metrics around the NTS. All of this requires integration with all the other different entities and relationships that exist. We have made a massive investment in our own context layer. We took a similar approach that we had for Trino. Trino is a federation layer. and we do not recommend you move your data to a centralized location. We recommend, because there's this entropy loss, we recommend you keep your data close to a source and we will enable the querying of that data. Similarly, we're taking this approach because we know your context lives in 10 different places. You may be using a certain catalog for understanding your data. you may be using a certain other service where you have your business rules and policies. One of our customers uses a very homegrown service where they have information on how to process returns. That's like if you talk to an agent and you're saying, I want to return this product I just purchased, agent needs to know how to process returns. And in order for this to do that, it needs to know, what are the business rules? What are the policies? And agents now need to, in order to process that return, it has to talk to multiple data sources. Who's this customer? Are they a high value customer? How does it understand it's a high value customer? So this requires access to so much data, just like agents require access to so much data. Now they require so much context to be able to make accurate business decisions. It's not an easy problem. It's a very complex problem because context harnessing, and presenting it together to agents along with the data. I think that's where the next billion dollar companies will be created. And we have a pretty solid start. We had a moat in data federation, and we have a moat in context federation as well. So that's where we're investing in our stack. We're helping our companies move at the speed of large language models.

Conor Bronsdon 51:17 I think that is the right approach today. Dr. Tinder, this has been a fantastic conversation. I've really enjoyed it. Thanks so much for coming on the show and super excited to continue this over time. Where are the best places for folks to follow your work and follow what you're doing at Starburst?

Jitender Aswani 51:33 Yeah, I'm on LinkedIn and follow us at Starburst also. Just go to starburstdata.com. You will find us. We have quite a few engineering blogs that talk about context. In fact, there is a a pretty long 56-page blog on how we are investing in context layer at Starburst. So just, I would say find us on LinkedIn. And yeah, Conor, thank you so much for having me here. I really enjoyed this conversation. I think this is timely.

Conor Bronsdon 52:05 Yeah, I would love to continue this. I think we just started scratching the surface of what we could have covered here. There's so much depth here, and I really appreciate you bringing it to the show. Everyone who is tuning in, thank you for joining us today, and make sure that you've subscribed. Check out chainofthought.show for all the details, episode transcripts, and much more. And subscribe on your platform of choice, and if you're on YouTube, turn on that notification bell. Thank you all so much, and we'll see you next week.