Apple Podcast Link
Inside the AI Infrastructure Race features Corey Sanders, Senior Vice President of Product at CoreWeave and former Microsoft executive, in a wide-ranging conversation with James Barrood about the technology powering today's AI revolution.
Corey shares his journey from growing up in New Jersey and spending more than two decades helping build Microsoft Azure to leading product strategy at one of the world's fastest-growing AI cloud companies. Together they explore how CoreWeave became a critical infrastructure provider for many of the world's leading AI model builders, why enterprise AI adoption is still in its early innings, and what it will take to power the next generation of AI applications.
Jim Barrood: Hey, friends. Thanks for joining me, Jim Baroud, to hear a few insights from leaders who represent our innovation ecosystem. Today's chat is with Corey Sanders, the Senior Vice President of Product at CoreWeave. He was previously at Microsoft for over 20 years
Corey Sanders: I grew up in New Jersey. I grew up in Parsippany which was great.
Went to high school in Parsippany. And when I graduated high school I went I got lucky enough to get accepted into Princeton, so I went to Princeton, so stayed in the state, which was great, actually. Stayed in New Jersey. And my original desire, actually, when I joined...
went to Princeton was that I was going to be an aerospace engineer because I really... I wanted to go to space ironically which we can maybe talk about if those skills could apply to me later in in, in my journey at CoreWeave. But I took my first computer science class and was in love, and so I ended up just immediately deciding I wanted to be a computer science engineer.
Did a lot of fun stuff. Princeton was great for... I studied Japanese, I studied a bunch of Shakespeare. But but, I computer science was my journey. I, again, lucky enough to get an internship at Microsoft and then started full time after college. I worked at Microsoft for 20 years.
And so I actually moved out to Seattle did my New Jersey hiatus for about 10 years out in Seattle and worked on various jobs. I worked fixed bugs in Windows. I worked on the original product of Azure, which was really awesome. Did that for about 10 of those 20 years, so that was a great experience.
I worked in the field, so I did some sales which I... turns out I wasn't very good at, so then I went back and did more product work before then I left. And after leaving I ended up getting to know one of the co-founders of CoreWeave pretty well. We met at Roots in Summit which is where I moved back to after returning back from Seattle.
And he ended up saying, "Hey, do you want to come and work at CoreWeave?" And I said, "Sure." I'd love to go think through product and strategy for the company. And os- offices in Livingston easy commute and great to great to be able to have the opportunity to work here.
Jim Barrood: Oh, that's a great, that's a great journey. And, Yeah ... just the fact that you went out west to Microsoft what a great experience, right? And then to come back to Jersey is is a wonderful story. And Summit is a wonderful town, as I
Corey Sanders: love Summit. Love Summit. Yeah. It's great.
It's a lot of fun.
Jim Barrood: So things are moving at light speed. You've been on this rocket ship the past few years, so tell us, what's it feel like, right? For those of us who are not in the thick of it, what's it feel to be working at one of those rocket ships?
Corey Sanders: Yeah. Yeah, look I would say we are at one of the biggest technology shifts in decades a- and we're in the middle of it, and CoreWeave is in the middle of it, which is really fantastic.
It's, it is such a, a, a awesome experience to be able to be designing and thinking through the products that will make AI applications easier to develop, easier to deploy. And that's really where I think, CoreWeave is really growing into. Obviously its history starting off with some of the co-founders working in crypto mining and now to the point where, we are the AI cloud, right?
It's offering a bunch of AI services. And so being at the forefront of delivering those capabilities is just really invigorating. It's just so exciting to see what customers are going to build, what they're going to do. And it brings me back actually to my early days at Microsoft, right?
My early days at Microsoft when Azure was coming out, one of the biggest things with our focus there was which I think is any sort of platform offering, you think so much more about what are the customers going to be able to do with what you're building than necessarily what you're building it directly, right?
Like it's, there's no end consumer working on CoreWeave, right? Like it's all through someone else, right? But so then it's all about how do you make the platform such that those end customers are able to go do their amazing work, right? And with CoreWeave today, it's primarily 9 out of 10 of the top model builders all use our platform.
And so like what are they doing? How are they changing the world? C- Cursor being a big one, like how are they enabling sort of people to think differently about coding? And so those types of customers even down to like enterprise, Mercadona Libre and thinking through their entire retail business that is what I love.
It's why I feel like platform is always the right place for me because I get so inspired by other people's inspiration and other people's creativity, and I'm just there as a tool to make it easier for them to get that done.
Jim Barrood: Yeah, for sure. And you've seen a lot, right? Obviously, with Microsoft, you saw their investment in OpenAI and, how that's developed and, we could probably spend another two hours just on that alone.
Corey Sanders: We could spend days on that, Jim, if we wanted to.
Jim Barrood: Can you just maybe opine on that just a little bit on, on, on that sort of- That development which kicked off this race, as I mentioned, and this- these advances that are going at light speed, and has that developed the way you would have predicted, or is everything just such a surprise?
Corey Sanders: Yeah. Yeah, I had... I did have the benefit of actually meeting OpenAI the OpenAI folks before actually the, the ChatGPT explosion because they were obviously, they were deploying on the platform, and I had some engagements with them on making sure the platforms delivered what they needed in those early days.
But I would say I was definitely not on the forefront of that of that explosion. But once it did happen e- everything at Microsoft changed. And I would say the world, right? But but living within the microcosm of Microsoft, it was this fundamental shift of every single application, every single service was, "We're going to shift how we think about this in this new era."
And so that was again, what, again, what a great experience, and I was a, a key component or I got to, the benefit of being a key component in thinking through how are we going to deliver all this infrastructure? How are we going to deliver all these AI services? And and that translated even into now what I get to do here at Corey, which is thinking through that, but in some ways even a higher level of vertical integration, right?
How do I think through the full metal to job technology stack to be able to optimize the infrastructure underneath so that we're building what we need to and not anything more and still make it super easy for people to be able to build because this stuff is hard. That would probably be the biggest thing that shifted differently than I expected.
The initial explosion of GPT and the, the market getting fairly crowded in sort of model creation and which model's best and then the open source model market I think I, I feel like I expected that. I expected a little bit of maybe more consolidation by this point, but I expected a lot of this kind of ongoing fight over whose model is best and for what.
The part that I've been surprised by, to be honest, Jim, is, is- The complexity and challenge for the broader base of customers and enterprises to use it, right? I think, you look around at where AI is actually taking off, and yeah, sure, everybody's got an agent on their website, right?
A- and so that's expected. But the actual integration of AI deep into the product from every single company in the world I think is still pretty slow. We're still, we still haven't seen that happen, right? And and I do think a part of that is the capability, the m- the a- approaching the, the concept of continual improvement.
We call it an AI loop here at CoreWeave, and the services to be able to deliver upon that. And I think that's going to be a big differentiator in thinking through fully from the GPU to those managed services and how customers can get the benefit.
Jim Barrood: A- and just for those folks who don't know what CoreWeave is, can you give us a sense for- What you folks do and what your footprint is now, and where do you see the company in five years?
Corey Sanders: Yeah, absolutely. CoreWeave th- we c- we talk about CoreWeave as being an AI cloud, right? It is a, a large scale provider of infrastructure to services. We have more than 50 data centers worldwide so a very large footprint. Pr- a very high amount of that is GPUs, but it certainly includes a whole bunch of storage and s- and CPU as well and then a bunch of the services on, that sit on top including sort of development services like our weights and biases, tooling that allow people to build models on top of that infrastructure, our inference service that allow people to run their applications on top of our infrastructure.
And so really in some ways the best way to think about it is like any other big cloud that also has a whole bunch of these data centers except our focus is exclusively on AI applications and AI workflows allowing us to, in some ways, shed some of the legacy of non-AI workloads and really focus on delivering upon this new generation and that's why I think CoreWeave is so well-positioned with this technology shift to be a central point in delivering these capabilities.
Jim Barrood: Got it, and five years from now, where is CoreWeave?
Corey Sanders: Yeah, I think, look, the growth has been stunning. And so I think with that continued trajectory, certainly the growth continue, will continue to be stunning from just an overall infrastructure provided, and a big part of that is just the demand has been insatiable, right?
It's, it's we certainly don't build data centers without customers screaming for the capacity, and those customers being, OpenAI or Anthropic or Meta or Google or so on. And so that demand continues and so we will continue to deliver upon that capacity. And then I think we will continue to expand from our services in being able to deliver easier-to-use deployment and continual loop or AI loop development processes.
I think as we see the rest of the enterprise market and really just the rest of the application market start shifting towards mission critical apps always have some AI if they're not primarily AI, we will start seeing a shift a greater shift towards those higher level services in their usage.
And I think over five years that becomes a predominant component of our business.
Jim Barrood: Oh, really? Oh, okay.
Corey Sanders: That's
Jim Barrood: fascinating. What about, we hear- Some flak and opposition to data centers, right? And I know there's some things that people get wrong as far as the water usage, which isn't as nearly as much as people say.
What is... Can we dispel some or explain some reality here so that people can get a better idea- Yeah ... of what the challenges are?
Corey Sanders: Yeah, absolutely. Yeah, and look, I think, Jim, there's the, the biggest complexity with some of the concerns around data center is there's not a single data center provider, right?
There's a, a bunch who are building these, these, this infrastructure. And just like anything with this type of approach the s- I would argue the bad actors are the people who are not taking into account enough of the community concerns and the community focus end up creating a shadow across the across the, the full list of providers.
And I, I'd start off by saying I totally understand and appreciate the anxieties that people have towards the risks or the challenges of this infrastructure. But then to your point about dispelling the myths I think, a- and I'll speak explicitly to how CoreWeave approaches it as a key factor.
From an... the biggest sort of concerns I think that come out energy being a big one and sort of energy pricing and one of the biggest things that that we've done is a real focus on making sure that we are investing ahead of those energy prices, right? The opportunity to advance and improve the t- the electrical grid in this country is real, right?
I think it's a, it's an important need for us without even the data center expansion. But then the sort of opportunity for Corey to take a big position in helping to expand that, and that sort of being a part of the investments that we make in many of these locations is part of the investment that we make to expand and improve the grid such that the residents aren't bearing that, that burden.
And of course, we don't have anything to do with the pricing of the electrical grid, right? And so it is that they're typically regulated and controlled and operate in that way. And I do think that is an important consideration and an important place, I think, for us to constantly think about how to improve.
Again, the bad actors point, make sure that the, the providers are supporting the infrastructure improvements as, as required in some of these build-outs. You already mentioned water, right? We use a fully closed loop system which not all the providers do. But so we have basically an initial intake of water.
It's about equivalent to maybe two days of watering a golf course. And then we're done, right? We don't take any more water in other than just the people who are, going to the bathroom or drinking water and running the facility, right? So there's no additional water that comes in. And then noise, right?
I think the other thing is we do a lot of work to be able to shield communities from the noise and even just the steady state of noise is quite low. And about- the regulations in the state in New Jersey specifically, which I just happen to know particularly well, right?
The low end of the noise is around sort of the the hum of a refrigerator. And then the high end of the noise is maybe a, a bustling office, right? And with the right protections and the right sort of controls in place, we feel like, again, good actors can help avoid the impact to the community.
While then also bringing jobs and bringing education. One of the things that CoreWeave has just recently launched, CoreWeave University, where we come in and actually educate people in those districts on how to operate the environment, right? So we actually have a program where we'll hire from local community individuals, and we'll train them and teach them on the job a-and and make them capable in this new set of skills in operating and being a technician in the environment.
And I think there are benefits that are missed, and I think the impact is oftentimes i-misconstrued, right? I think is maybe the best way to put it. And but then all of this, of course, is in the deep awareness that communities have a right to raise concerns, and we should be listening, and we should constantly be getting better.
That, that is the beauty of the type of, systems of government we have here. And I'm super supportive on the feedback and making sure that communities have this voice and that voice is heard.
Jim Barrood: And thank you for explaining that. That was really helpful.
And regarding the communities, it's so impressive, the CoreWeave University. I hadn't heard about that before. But I know you're very involved in New Jersey, with the New Jersey AI Hub.
Corey Sanders: Yeah.
Jim Barrood: Talk about that, and maybe what you do in other communities around the country and maybe internationally as far as engagement to really help further, Yeah
retraining, upskilling startups. The explosion in startups is something that I think you guys are sort, supporting.
Corey Sanders: Yeah. That's right. There's a few things to say. First of all, with the New Jersey AI Hub, it's been... I've, I actually get to be a, the board member from CoreWeave as part of the hub, and that's been such an honor.
It has been so wonderful. In fact, you can see behind me I've got the orange ribbon for the, for when I cut the ribbon which I hold in sort of this prominent location behind my camera here. I should really get it framed or something. But it's been such a great experience. Obviously getting, Microsoft being one of the other co-founders of the hub and people that I used to work with, and so that's been really fun to bring those connections in.
And then, obviously getting to work with the EDA. I think they've been just I- so focused on New Jersey development a- and to your point, the focus on training and focus on being able to enable this startup community, this innovative community, and to entice them to come to New Jersey, which I think is great.
And we've got some amazing, the, the State of New Jersey has such some amazing enterprises and a- amazing sort of technology sectors whether it be pharmaceuticals, whether it be communications, whether it be energy itself. And so I think being able to now entice some of this innovation, some of these startups who are leading the way with AI, whether on the plat- AI platform offered by CoreWeave or other platforms is not the point.
Being able to create this sort of a- additional ecosystem here in the state is so great for the state, for CoreWeave, for everybody. And that's been a big factor and really the focal point that I've had with the AI Hub. Obviously, it does- The Princeton connection's really great, and the Princeton focus has been also around enabling this within universities and schools, which I also find really quite compelling.
And so for me, I think that combination of the university, the industry and a- and the education and bringing all of that together in a single emotion has been really quite inspiring. And to your point, in other communities there's a couple things there. One, the New Jersey AI Hub has actually become the model that we now go out there, and people ask us w- you're coming here, you're building here.
Tell us a little bit about how you can kind of engage the community." And we always point to the New Jersey AI Hub as being a good example of something where you've got public, private a university coming together in this sort of powerful three-way thing sort of government involvement as well.
And we use it as the example of where that can really drive innovation. And, we go to different communities, and maybe one community is a has a big automotive market, and so that would be where the automotive AI companies should go, right? Versus pharma, which should come here. And there's already this sort of regional opportunity that I think we could just tap into versus today it feels like all of the AI sort of go to one place, or all of the technology starts to go to one place.
And I think the opportunity is quite real to bring them into their sort of centers of excellence around the country and I like the idea of Cori playing a key role. We've also launched a program called Cori Ventures, where we are directly investing in startups around the country and around the world in who are doing AI work and interesting AI innovation.
And so that's a, a separate effort. It's not New Jersey specific but it is a- an effort to bring some of that innovation to life.
Jim Barrood: That's great to hear, and we'll put that link in the show notes if that's okay so people can learn about that. So are you... So are you domestic or do you have locations aro- around the world, internationally?
Corey Sanders: do. Yeah, we do. Yeah. Yeah we have a bunch of locations in Europe and Canada, and and a lot of a lot of international expansion underway. So yes, it's it's been... That's been a, that's been a very fun and exciting and interesting part of the role as well.
Jim Barrood: Yeah, no, I can only imagine.
Corey Sanders: Yeah.
Jim Barrood: That could be a another show. Speaking of international and trends and things, what are you seeing out there as far as, And we see the fight for the frontier models, right? Each one getting better on a week by week or month by month basis, right? And that's interesting. And then we see a lot of stuff moving to the edge moving to devices with or potentially moving to more to devices.
And then you have other actors around the world who have lower cost models, open mo- open, weight models. And so I'm just curious- Yep ... how does, how do you see that from your perch?
Corey Sanders: Yeah. Yeah, I mean I th- I think, We are definitely seeing that inference demand, really an inflection point probably starting at the beginning of this year, where inference demand has become unrelenting.
It is so indicative of the that the application era of AI has come, right? I would say up until January this year, it was the research era, right? It was the sort of... and maybe the consumer era, right? People engaging with their chatbots and so on. But we are I believe we have entered into the application era, where services and solutions are being actively built and deployed.
I also think with that comes a level of complexity. And so a little bit to your point I think the idea that applications are going to have a single model that they use for everything I think is actually a false expectation. I think we will see applications be built with a complexity of models.
You'll use 30 or 40 different models to deliver one, relatively complex mission critical application. And in sometimes, in some cases, you will use the frontier models, right? Of which again many of them are running on CoreWeave infrastructure to serve those experiences. And so you'll use those for some aspects, let's say your deepest analytics, perhaps.
But then in many of cases, you will want to use perhaps something that maybe you can improve yourself, right? You may want to use, like you said, an open source model that then you want to make specific to your data, right? So train specifically to your information or tweak or use very s- very bespoke models or very particular models for specific parts of your workflow.
You're doing gene folding, it may turn out that it would be better to use a model specifically focused on gene folding than just using the frontier sort of does everything Swiss Army knife model, right? Whether it be for performance or cost or experience. And so I believe the combination of these models are all going to come together into single apps.
And so there's not... Like I, I think there people are constantly looking for who's going to be the winner or which models is going to win or is open source going to win or is... and I actually, the answer is yes to all of it for me because I think it will be a combination of all those things. And this is where, when I think about where CoreWeave and where I want to go add a lot of this value is helping customers get there to their application, right?
The step between I want to do this and now I'm going to build it this way is a, a big leap right now. I think it's very hard for people to think through. It's like what model should I use and how do I get there? And just like any other application development, I think it's going to be constant iteration, continual loop.
And so how do we build the tool so that you can understand how your model's doing in production, understand when you're evaluating these six different models which one gives you a better cost even if it may be worse results. And then if it worse results, what part- part of my app can I deal with worse results because I don't really care, versus which parts, no, they better be the right results, and so I'll take it maybe a little bit more expensive or a little bit slower to get the better results.
And so this type of evaluation and analysis these are all tools that we offer and capabilities that we offer as part of the Core platform. And so I think that's where we'll see a lot of really good innovation. And right now I think it's just a hard bridge. It's a hard, it's a hard step to get there for a lot of people who didn't grow up in this world who aren't working at those frontier labs and aren't experts at this.
And that's, I think, the opportunity for a company like like CoreWeave to be able to create that bridge and make it just much easier to get to that outcome.
Jim Barrood: Oh, wow, okay. I'm glad you explained that. You did mention inference, and for those folks who don't know what inference means, can you just explain that in plain-
Corey Sanders: Yeah, sorry.
I should've explained that. Yeah. Yeah, see, the, in the AI world in some ways it's broken down into two big buckets. One is, one is training, which is taking a, either starting from scratch or taking a model and making it better. And that's taking a whole bunch of data and feeding it in and having it learn, right?
And we actually model a bunch of this and obviously I'm going to prove my depth here. We model a lot of this i- in a very similar way to the way brains work. And in fact, you can think about it the way brains work is you have neural pathways, billions of them in your brain, and the pathways that get hit the most get stronger, and the pathways that get hit less get weaker, and that's how we remember things.
That's how we learn things, right? We remember we do math enough that those pathways get strong and so then we're able to just instinctively go down that path and say four plus four that's eight," right? And so it's a very similar mental model in how to think about how training works in AI, right?
It's basically it runs through billions and bil- trillions even of sort of iterations, and that burns these paths or weights, as people like to call them, and that then becomes what path is most likely to be run when you ask a question, right? When you ask a question, that's the path that, oh, you asked about sporting events.
Okay the most likely path now for sporting events right now is let's talk about the World Cup, right? And so that becomes a, a similar sort of it's been burned in. So that's the training side. But then the inferencing side is actually taking those weights, taking that burned path, and then running it, right?
And so when the question comes up, "Hey, what's the, what's, wha- how is the US likely to do tomorrow," right? And so that then becomes the US what could that mean? And so it runs the paths that most like- oh, the US today, based on everything that's happening on the internet, most likely means the World Cup, so it means the US national team.
Okay, are they playing tomorrow? Okay, who are they playing tomorrow? And it then follows all of that pathway that had been burned in the training and comes to the most likely answer as part of it. And I'm, by the way, I'm way oversimplifying. There's tons of math and multidimensional metrics and so on and, but that to me...
So that's the difference between training and inferencing. There's a bunch of different ways to do inferencing, right? You can do it as an agent that basically will, take your calls and go run some code and maybe call some tools to help, like a search tool to look up, "Hey, what's interesting right now?"
But then you can also just run it as like a, as a simple sort of command and response. And then there's a lot of different ways to do training, right? There's the original base model training but then there's also a concept called reinforcement learning, where it's very similar even to how we learn sports.
You do it enough and your muscles get memory, and that's one typical type of train. So anyway, I've gone off track, but that, hopefully that helps, Jim.
Jim Barrood: That, that was super helpful. Thank you so much. This has been a great conversation Corey. Thank you for joining me. We usually do Just One Thing's short clips, so one prediction for, the AI ecosystem over the next couple years?
Corey Sanders: Yeah, given we're seeing this inflection point from January, I think every app and every experience that we have will have AI integrated within the next two years.
Jim Barrood: Okay. What about are you concerned about data centers in space?
Corey Sanders: Concerned? I think, currently the data centers have a significant operational burden, right?
They do require quite a few people. As I mentioned I did want to be an astronaut, and I would be willing to maybe spend some time working up there. I'm not sure you want to see me in a space suit though. But, I do think right now there, there's enough power worldwide that it's not a top priority.
But, certainly as the sort of overall market shifts and as demand continues to grow we don't discount any ideas as bad ones. But certainly a lot of things would need to change to make it, to make things work in a space environment.
Jim Barrood: Got it. One last thing. When do you think AGI will
Corey Sanders: come?
Oh, gosh, that's a tough one, Jim. Part of it is also I find that there's a lot of different definitions for AGI, which, again for your audience is just gener- artificial general intelligence, basically. Y- I think right now the focus and research has really been on sort of incremental improvements and then task-specific AI opportunities.
And I think the combination of these are going to deliver so much value that at this point I don't see it in the near term. I think that there's, in some ways a research step that's required that probably requires us to think about the entire model differently, but I'm not sweating it because I think the amount of improvements we can make to our day-to-day lives with the technology as it continues to evolve right now is mind-blowing.
And so I am just so enthusiastic about what customers can do with what we've got today and will have over the next six months that to a certain extent I'm not sweating the timeline for AGI. If it happens to be a year, great, like I look forward to then what customers can build on top of it. If it happens to be 10 years, like I, I still think the type, the, the world in which we live is changing and I think it is so exciting to get to be able to be on a, in the front row seat to that.
Jim Barrood: You bring up something really interesting. Should we assume that if when there is AGI that the usage, the demands will be greater on data centers?
Corey Sanders: It's super hard to say, Jim, because it all depends on, so much of this also depends on just the way that the research goes. I think research on optimizations, research on being able to run with a lower infrastructure footprint to accomplish the same task has been really exciting research over the last year, right?
And we're seeing significantly scaled back models that can do a lot as compared to what, two years ago, right? And so at a point in time, at the moment that AGI comes out, it will a- almost certainly be an uptick in the sort of raw amount of infrastructure required. Will it be able to then paralyze more because of its capabilities?
Maybe, right? And then will there be a path of optimization? Certainly. And if AGI lives up to its expectations, maybe it will do its own optimizations, right? And I think there's a path there to think about. And this is going to be a constant pull and push on d- driving the optimizations while also making sure that frontier research continues to develop.
And again, I think both are going to deliver huge value to all of us.
Jim Barrood: That's great. Thank you for that clarification. This has been a great conversation, Corey. You bet, Jim. We usually end with a quote or a saying or a poem. What do you have for us?
Corey Sanders: Yeah, absolutely. I... there's a quote that I really and it turns out there's some debate.
I think Harry Truman said it, but then Ronald Reagan also said it, so I'm happy to give both credit and try and be very bipartisan here in my in my response. But the quote goes something like this. "It is amazing what you ac- can accomplish if you do not care who gets the credit."
And I believe it is such such a meaningful statement over the challenges of human nature. Like I... By the way, this is not something that I feign to totally live up to. I love getting credit for things. Let's just be clear. This is not like I, I have perfected this.
But I want to. This is a, a way that I want to live my life, that I want to be okay that anyone can get the credit and because then you do reach this point where you can accomplish so much more if you really don't care and I find it such a a strong statement to what really matters in life.
And but it's hard. You, hu- y- humans want to get credit. You want to be... You want to get the credit for things that you did well, and sometimes even things that someone else did well. And so that's a challenge, but that's a quote I love to say.
And again, I think both Harry Truman and Ronald Reagan said variations of it.
Jim Barrood: Yeah, no that's a great way to end. Thank you so much, Corey. I really appreciate you doing
Corey Sanders: this and- Thank you, Jim ...
Jim Barrood: look forward to collaborating with you in the ecosystem.
Corey Sanders: I appreciate it. Yeah. It's been fun working with you, and thanks so much and happy to come back and talk more whenever you want.
