We recently hosted Modular Co-founder & CEO Chris Lattner and SambaNova Systems Co-founder & CEO Rodrigo Liang in conversation with Mayfield partner Vijay Reddy at TechCrunch Disrupt – here are some key takeaways from the event as well as the full transcript.
Vijay:
Hello. Hi. Thank you all for coming to this panel. I’m super excited to introduce both Chris and Rodrigo, and also to learn more about their founder journeys. I think the goal of this meeting was mainly to learn about how they went from inception to iconic. So let’s just start off with a brief introduction of the panelists.
Maybe I’ll start with introducing myself. I’m a partner at Mayfield. Mayfield is a 50-year-old fund. We are known for our people-first philosophy, and we invest at the inception stage. We just announced a new $250 million AI fund investing in AI start companies. So when you look at founders we invest in, they fall under three broad categories.
Some are AI researchers who are pushing the boundaries of AI. Some are product builders who have built exceptional products at scale at large companies, and some are founders who have gone on to build companies before. And today I have two of them who embody all three of these categories. They’ve been exceptional researchers and technical experts. They have built iconic products in their previous companies and they’re repeat entrepreneurs. So it’s very rare to find a combination of those three.
So thank you Rodrigo, and thank you Chris for joining in. I’ll briefly introduce yourself the way I know, and I’ll ask you to maybe introduce your companies and also talk a little bit about yourself if I missed something. So Rodrigo here, he’s the CEO of SambaNova like I mentioned. He was previously Senior Vice President of Oracle, where he built 12 to 15 processor lines. Before that was VP of Sun Microsystems where he came from after acquisition of Afara. And I don’t think people understand how significant those product lines are. Maybe you can introduce when you talk about it.
Chris is the CEO of Modular AI. He’s going to talk about the company. Chris built four or five programming languages. I’ll start from reverse order. It’s Dojo, Swift, MLIR, LLVM, and I missed a few in between. And these are foundational compiler and software technologies. And Chris went on to build TensorFlow. And he was a key member of TensorFlow team at Google, and autopilot at Tesla.
So an exceptional engineering track record and product track record. And previously he was the president of Sifive, which is a RISC-based company. So it’s my honor to introduce them both. And maybe I’ll start off with Rodrigo, if you could just talk a little bit about SambaNova. You had some great, great announcements yesterday. Thank you.
Rodrigo:
Yeah, no, thank you. Thanks for having me on stage and thank you to Vijay for inviting us on this panel. So as Vijay said, I’ve been in this high performance infrastructure business for enterprises for more than 25 years and building hardware, building software that go into mission-critical applications.
And we’re in this just amazing time where AI is starting to take over the world. Chris and I were just talking about ChatGPT. We’ve both been doing AI for a lot of years, but in the last six, nine months, suddenly the entire world woke up to it. And so it’s just such an exciting time.
With SambaNova, we do infrastructure. And so we build chips that, on one end, compete with NVIDIA, and on the other end we build up to foundation models that deploy in the enterprise. And this is what one of these looks like.
Basically one of these is equivalent to… Actually it outperforms the H100, but it’s effectively what people are hunting around for. If you want to run these GPT models, you need chips, and that’s the end of the story. One of these systems can now power 5 trillion parameter GPT models. And we actually have a demo up and running on a trillion parameter model that people were seeing yesterday.
And I’m really excited about that. At the end of the day, we think of this as a generational opportunity for all of us here in this community to build applications, build software, build infrastructure that is going to last for years. It’s going to last for decades. And I’m so excited to be part of it.
Vijay:
Chris and I were just joking about this. That was a H100 chip that’ll probably be worth $100,000.
Rodrigo:
It’s hard to believe that you think about cars in relation to chips now. In some states even houses, right? These things are just amazing, the ability of actually taking these things and turning them into global value. Opportunities to actually create applications, create solutions, create value that’s globally accessible. It’s just amazing. And so there’s a reason why NVIDIA’s a trillion-plus valuation company. But look, there’s a lot of space for all of us to actually innovate and try to support the market.
Vijay:
Chris?
Chris:
Cool. Well, that’s pretty impressive. I work on the software side of things. I mostly turn zeros into ones and ones into zeros. My background goes back through developer tools. I grew up at Apple building programming languages, compilers, systems. I really care about and love developers. And so I’ve gone through a whole journey, my own AI journey of trying to understand what this is through the applied space, through the developer attentive flow, brought Google TPU to market, worked on a whole bunch of different parts of this problem.
And I didn’t realize that software is a major part of the issue. And what’s happening today is that so many developers who are building AI for their products are struggling with complexity. There’s just so many of these systems. They don’t work together. TensorFlow, PyTorch, we love them both sort of, but they’re kind of dogs and cats and fight each other.
And then when you get to deploying things, it becomes this gigantic mess. And so what Modular’s doing is we’re entering the space and saying this is no longer research. The infrastructure itself should not be research. What we’re doing is we’re saying, look, we’ve built all of these systems across many, many years. Let’s go back to the fundamentals and rebuild them from the bottom up.
And where many people are saying, let’s fix AI by adding layers on top, we’re saying let’s start at the hardware and build those fundamental software layers in a super-accessible and useful way so that people can use them and take advantage of this amazing silicon without having to know all the details. And so that manifests itself in two ways. One is a new programming language mojo. You can think of it as a Python-plus-plus, which gives Python programmers superpowers where it’s very compatible with Python and the entire ecosystem. But it scales all the way down to running it on something like that (SambaNova chip), which you can’t typically do with Python.
The other is an AI engine, which is a drop-in replacement for TensorFlow and PyTorch. This meets you where you are so you don’t have to rewrite all your models. But it gives you hardware abstractions. You can get to all the different places you want to be along with performance, programmability, and a whole bunch of other features. And our hope is really just to help people. Because in my experience, people are struggling with all this stuff and people want fewer things that actually work better.
Vijay:
Thanks Chris. Just as a show of hands, how many here are either founders or want to be founders? So I think we have the right audience here. The panel discussion is about inception to iconic. So I want to take us back to inception. You both had successful products in the market, which millions of people, if not more, were using. What made you get up one day and say, here’s a problem I see. I want to go solve it. Take us back when you decided you wanted to start a company leaving the safety of your previous job. I think a lot of people here have really good careers, and at some point they need to take the leap. So what was your journey like?
Rodrigo:
Well, I’ll say mine’s easier because I didn’t wake up one day and decide that I was going to do because my two co-founders are Stanford professors who’ve been working on this for quite a while. And like Chris said in the computational side, I think it is very clear what we’re feeling now, is that the brute force way of running these large, large models just by aggregating more chips and aggregating more resources and aggregating more people to brute force our way through it is just not sustainable.
And so they started thinking about it and did a bunch of work at Stanford and a lot of research there. And they came back with the conclusion that not only do we need to rethink the stack as Chris is talking about, we also needed to create a new piece of silicon. A new piece of silicon, which sent shivers up my spine. Like I said, the last startup that I did was acquired by Sun and within a few years became 100% of everything that Sun Microsystems shipped.
But it was a lot of work. So one thing I told them – and Vijay was with me early on in the journey – I told them that if we’re doing another chip startup, the first requirement is you’re not going to run out of money. As a startup, you just got to make sure that in a very capital-intensive venture, you don’t run out of money.
And so sure enough, in the first three and a half years of the company, we raised $1.1B in venture capital. We’re in a great position to be able to compete. It wasn’t what I woke up to do, but it was certainly the contribution I had to that venture to say, look, we’re doing this. We’ve got to find a way to raise enough money so that you can actually sustain yourself. Because in this space when you’re doing core tech, you have to show up over and over and over again. And we want to make sure that we actually have enough capital and enough people to be able to compete.
Chris:
My journey is a little bit different. So I really created Modular in the middle of 2021. That’s when my co-founder and I started talking about it. If you remember, 2021 was a very frothy time, particularly in AI. And so I was in this position of saying, okay, do I want to go be head of AI, whatever, at some big company somewhere, or give up the predictable and understandable path that is also fraught with politics and other challenges, to go build something from scratch?
The challenge there is that the ROI, the opportunity, has to pay itself back better than taking the easy comfortable job. And so what happened is my co-founder, Tim Davis and I spent a long time talking about how we approach the market, how do we do something meaningful? How do we understand the different challenges that people have? How do we do something that’s differentiated? How do we do something that’s not going to be fast-followed? How do we create core value in the world? And similar to you, Rodrigo, we realized it’s not going to be cheap.
And so what that really did is it shaped our entire philosophy of how we hire, how we build, how we pay people, how we invest in the technology and build the differentiation that we’re trying to create in the world. And so a lot of that was that soul-searching, which worked out really well when things changed in 2022. Because starting a company with a business model turns out to be a pretty useful thing and not always something that all AI companies do.
Vijay:
I think these two companies have been lucky enough raising a billion. And you just raised a hundred million Series B, or A, whatever you want to call it.
Chris:
Or second, right?
Rodrigo:
However you want to label it.
Vijay:
So you decided which hill you want to climb. How did you bring on the first set of co-founders or the first set of the team? A lot of people here I think know which hill they want to climb, but is it going solo? Do you go and find people in your past or do you go find new? Maybe if you could just talk us through your first few hires or co-founder journey, that would be helpful.
Rodrigo:
Yeah, and this is true for SambaNova. It’s also true for the previous companies that I’ve done. I strongly, strongly advise you find folks that you really enjoy working with, you really like working with. Any startup is full of surprises, full of twists and turns. And you want somebody that’s going to collaborate with you. You want somebody that is willing to go through the ups and downs with you. And so our strategy has been not only to just find those great people that are just experts and really, really proficient in their respective areas, but people that you just really collaborate well with, right?
Early stage, you just don’t have a lot of time to actually deal with the gears not meshing. And then it’s not just with the team. That flows right into investors, and your advisors, your community, because you’re surrounding people around you to help you. And so all those people have to bring their expertise but in a way that works with you and the company.
Chris:
I’d say that there’s a really big difference between co-founders and the founding team. And I think that that difference is really, really important. The co-founding team, for better or worse, can turn over, but you really don’t want to turn over a co-founder. That’s really a bad thing. And so for me, I’ve worked with my co-founder Tim before, and so we knew how each other worked. We’re also very complimentary. We spent a lot of time discussing and understanding how we approach things, how we build things to make sure that when we said go it would actually be a thing.
This is not something that I would advise jumping into lightly. But I’ll have to say, having led many engineering teams across many different organizations, there’s really something special about having a co-founder, right? There’s a partnership there that is not the same as having many other friends scattered across the organization. And having that soul bond I think is really quite amazing.
Now when we started to go recruit and build our founding team, it was actually pretty funny and pretty interesting. We had this whole idea that these teams of people we’d worked with before, would just follow us and go create the founding team for our new company. It turned out that zero people actually did that. And so we built our entire team organically from scratch.
And turns out that actually has, it’s challenging of course, hiring is always challenging. But it has an amazing benefit, which is that you’re bringing in people and you’re bringing them into a culture where they’re opting into things from first principles. And it forces you to communicate well, explain the mission well, explain why, what they get out of it, all these things that are really quite important.
And as a consequence of that, yes, we have a special founding team that was the early people that bet on us, some of them before we were funded. But we’ve built a much more consistent team that really works well together. We put a lot of energy into making sure people work together exactly as you say, because bumps happen. But I think that was kind of fortuitous in hindsight, even though it was challenging at the moment.
Vijay:
So you picked the battle. You found the team you want to go in the trenches with. And the next step is how do you build a culture around this company? And starting an AI company’s extremely expensive and you could easily use all your venture dollars. Billion is not enough to hire enough people here, right?
Rodrigo:
That’s what I said.
Vijay:
So how do you build a culture where you can attract top-tier talent into the companies, and they want to stay with you and not just go to the next startup? What’s some unique traits of culture onboarding at the companies?
Chris:
So for us, we started with a problem. The problem that we had is we had to attract these really specialized people out of really well-paying big tech companies. And if you think that hiring ML researchers is difficult, try to hire a compiler engineer. It is actually an order of magnitude more difficult. And then try to hire a compiler engineer who knows anything about ML, and then you understand what we’re talking about.
And so these people exist, but they have cushy jobs in many other places in the industry. And so one of the very first things that we did, again, we spent a lot of time doing homework ahead of creating the company and raising money and doing all the things that people think are glamorous. We spent a lot of time on culture. We wrote it down, we spent a lot of time debating about what kind of company do we want to be, published it straight on our webpage. And that continues to be a really important part of how we hire, how we run the company, how we operate as a group.
And then when you go to hiring people, yes, you need to raise enough money so that you can pull people out of the big cushy job they have, give them enough security, pull it back and develop the processes that they’re used to. So things like HR things and things that you may not invest in. That unglamorous work actually turns out to be really quite important or quite useful when you’re scaling up the team and you’re growing. I don’t know if there’s one right way to do it, but to me that culture investment is huge and it pays itself back even though it feels like you’re going slow so that you can eventually go fast.
Vijay:
I think as a former chip person, finding a compiler person who understands AI, it’s easier to find a unicorn.
Rodrigo:
It’s hard, but it’s a really good question. I’ll give you my pitch. Somebody comes in front of me, this is my company pitch. And I do it even today. So come join SambaNova. You’re going to work longer hours. Relative to your current job, you’re going to get paid less. Much higher risk, much higher risks, but you get to change the world. And people self-vote in. Some people don’t want those three things.
As a startup, I’ve always thought about this in terms of have we pushed the challenge far enough? And that’s important because if you haven’t, a Google, these monster companies are going to come, and within a few months, clobber you. So how do you park the company so that you push this far enough so that you buy your team a chance, but not push us so far where people are rolling their eyes, like, “I don’t know what they’re doing.”
And so that’s one of the challenges. But for us it’s always been about are you pushing to the point where if you’re successful, and you don’t have to think about 90% chance. 10% chance of success, great, but if successful, will you change the world? And if you are able to articulate that, are able to articulate that in a way that people come in and say, “Okay, now I’m able to do something meaningful. I’m able to actually have my hand in something that I can’t in a big company.”
I know. I came from Oracle, I came from Sun. Huge companies. And so if you are able to actually articulate something where if, if successful, you can change the world, a lot of those people with that mindset will come in and come in for that reason. And that’s actually incredibly aligning as the journey of the company goes along. You have to always keep in mind, we’re here to change the world.
Vijay:
I think your two companies in particular, there’s just to reiterate what CNBC said, are NVIDIA killers with your new chip.
Rodrigo:
No pressure.
Vijay:
No pressure. And Chris, for you, The Information just said that you’re going to take on NVIDIA on the software stack. So those are enough to attract a lot of people to want to go after the mission of a David versus Goliath story. Small guys taking on the big team.
And given that we’re at an AI conference, but we look at AI startups in general, we see a lot of startups using GenAI wrappers around products and calling them AI companies. And there’s fundamental companies which are pushing the boundaries of AI. And nothing wrong with either, but we tend to invest in the latter ones.
So we think that it starts with the team and the culture and pushing ahead. And it’s not just about hiring junior AI people, AI engineers, and calling it an AI company. So you are in the frontier of AI. Who drives the AI strategy? Are there different processes, is it different culture across the company, and how do you stay on top of all the recent advances? What’s the secret to your AI story?
Chris:
So you asked four things. Can I pull you back to that first one? So I totally agree with you that changing the world is super important. Because if you’re not on a mission that matters, why bother, right? And so for us, people want to paint Modular as an NVIDIA killer because I think that that is something that is just like this clickbait-y trend thing or something.
The way that I look at it is that the compute is fundamentally changing. If you go back to before Moore’s Law ended, single core, single thread performance was increasing very predictably and it was fine, but now we have things like that. Things like that are not programmed with the C compiler. Or maybe your people can, but that’s not the way that the world works, right?
Computers are getting more personalized. It’s wearable. It is specialized. Power matters. AI, networking sellers, video transcode, all this stuff is happening. And AI is the first example where we’ve gotten people to level up the programming model to the point where they’re not having to write for loops. That to me is something that’s really quite profound.
And so what we’re trying to do, and I’m sorry you’re pushing my buttons with the NVIDIA thing. NVIDIA is amazing. I hope you agree. Also, they built some amazing chips. And so the question is how do we get more people able to do more cool things with all that hardware, including NVIDIA? And sure, we can allow people to use other tech stacks and stuff like this, but the thing about CUDA is that very few people actually love CUDA. And so if we can level things up and get us more people to participate in this ecosystem, I think that’s just great for compute. That’s great for progress in general.
Rodrigo:
Yeah, I think when we’re looking at press and things that are said on CNBC, I think it’s great to have these little taglines. But we’ve always viewed ourselves as a company that’s trying to solve end solutions for enterprises. And so I do think that NVIDIA’s done a tremendous job building an ecosystem around them. Who doesn’t know NVIDIA? My parents are talking about NVIDIA.
But I do think that the world is going to be very expansive. It’s going to be layered. It’s going to be layered by what Chris said, these different tech stacks. And then you’re going to be also segmented by verticals and different people doing different things. Some in the core data centers, some of them off in the edge and by discipline. And so I think it is a great opportunity. We’re here, we’re looking at a very specific segment on a mission-critical AI for enterprises.
That’s what we play. I think there’s going to be lots of hardware, lots of software plays that will be just for that use case. There will be all sorts of other use cases where I think people will have a great opportunity to innovate, create, and compete for years to come. I do think that something that Deirdre Bosa at CNBC told me yesterday was look, investors are feeding startups a lot. And we’ve seen that not every startup makes it.
And so the question that also comes back is what is that secret sauce that gives us a chance? You can’t control everything. The markets will do whatever they will do. The competitors will do whatever they’ll do. But what is our core advantage that allows us to actually deliver value year after year after year? If you can hang on to that and be very sharp about that, and innovate and continue to compete on that front, you have a chance. You have a chance.
Vijay:
Maybe I’ll ask the second part of the question because I’m personally very interested in that. Who’s driving the AI strategy and what advice do you have for companies which don’t just want to build a wrapper but want to build an AI team around it?
Chris:
Well, so for us, I can’t speak in general, but we’re in the infrastructure layer. And so one of the things that’s really interesting is that in the infra layer you have many different skill sets and many different kinds of profiles across your company, even within the engineering team. Not even to mention the product and other kinds of roles you’ll have. And so one of the things that we encourage is a very open community of discussion. We have to use the Slack tool to do this.
But whenever new papers come out, new ideas and things like this, people will basically self-select all the noisy stuff happening on the internet, what’s actually cool. And then when they do that, well, then we can have internal tech talks and we can have other information sharing kinds of things because AI is almost too big to fit in any one person’s brain. And so I think that getting your team to engage in that, and allowing these bottoms-up feeder networks so that you can keep track of that I think is super valuable. I also happen to be a huge nerd, and so I keep track of a lot of it directly, but different people have different approaches.
Rodrigo:
So on the chip side, one of my co-founders, Kunle Olukotun, a professor at Stanford for 30-some years, he had a clear vision of data flow, what data flow can do. And so you do need very strongly technical people who can be visionary and help articulate to an entire company that hasn’t really gone through the whole technical process, to just decide this is where we’re going to go.
Same on the software side with ML, Chris Re, my co-founder, again, somebody that just has a great view of where the models are going, how people are engaging with AI. And so you do need the initial direction from your co-founders or whoever your technical needs are. And then over time we’ve transitioned to a much more internally-led and probably now even more customer-led, where customers are coming in telling me, well, we’d like to do this, we would like to do that.
We don’t want to put a trillion all at once. We want to add trillion parameter LLaMAs one at a time, and so can you build a chip that allows me to do that? And so we become very kind of customer-centric as far as where we need to change from our initial product so that we actually meet the customer where they are.
Chris:
One thing I’d add is that a thing I struggle with is that we have a small team. We have to be extremely focused. But AI doesn’t play by rules. And so there’s a certain amount of enabling people to be bottoms up and enabling new ideas to come out. Because you can have a plan, but when a new thing like ChatGPT or something comes on the scene, you need to be able to adapt. And so I think the AI, because it moves so fast, leads to a different kind of leadership in some ways.
Vijay:
So when we see a typical enterprise startup, an AI startup, the board level discussions are very different. So in the AI startup, the sales team, the marketing team is all plugged in. The way you go into sales motion is very different. And in a non-AI startup, it’s just one more point in engineering update.
Do you see that in your companies at the executive level where AI becomes a team across sales, marketing, support, functions, or you think it’s mostly an engineering? And maybe it’s your infrastructure company, but what advice do you have for startups to uplevel the AI conversation up to the CXO level?
Rodrigo:
Let’s see. Yes. So we work primarily on enterprises and so focusing on CIOs, CTOs, CEOs of many of these companies. And so a couple of good things. One, AI is so pervasive now. Everyone’s thinking about them. I saw enterprise hardware software for years and it’d be rare for the CEO to come and talk about product purchases. Very rare. And today it’s very common. And so I think my advice is making sure that you have a tiered articulation of the value for different levels of the company.
The CEO comes in, they’re not trying to talk speeds and feeds of how do you compare, what are teraflops, bandwidth. They’re not talking about that, right? What they’re talking about is what is the strategy for my company over the next five years and how do I integrate AI for these different things? That’s that conversation. But that’s very quickly followed by the next layer, followed by the next layer eventually to somebody who’s coming down. Show me how I program this thing. Show me what I have to change in my software to make use of your products.
And so I think just be prepared. And be prepared and be intentional about having those conversations because we, as someone would call circling the castle, these types of enterprise deals do not happen with one person talking to one person, handshake and checks get written. You have to talk to this person, then talk to this person, then get buy-in from this person. Then sponsorship for that.
Certainly you have to take the time to build support. Because AI is going to be such an important part of their company, and it’s such an important part of their strategy, that you want to make sure there’s buy-in, otherwise you’re in and out. So if you want to get in and stay in, you have to do that.
Chris:
I would say that our experience is quite different on the software side because of the nature of what we’re doing. The traditional way to start an AI software company is to, well first of all, ignore sales entirely. Ignore revenue and things like this, and then get up to the point where you have to do it and then try to figure it out later. When you do that, you naturally get into this mode of being an enterprise sales, big heavy sales team kind of organization, because you’re almost retroactively trying to figure out how to sell the product you’ve already built.
We started from a very different position. Our fundamental goal is to enable developers, researchers. Enable people building AI into their products. And so our number one most important thing is to build tools and technologies and program languages and all the different components that people fall in love with. If people will fall in love with it, well then you have what’s called product-led growth.
And this is something that’s seen much more in companies like Stripe or other companies like that, where suddenly the product, because people can recognize the value when they are using it, they’ll push it up into the organization. Now the interesting thing about that motion is that then suddenly you have to have this other part of the problem, which is as you’re saying, what level of the organization actually makes the decision.
And so getting people that are using your technologies to talk to each other within the organization, and having enough working with the teams, and figuring out who’s the VP or the CIO or whatever it is that needs to make the calls, is kind of a dark art. And this is where there’s friend networks, good investors can help, and other people like that can be very useful.
Vijay:
But I have one last question. So I think a lot of people, a lot of startups we have talked to and the founders, they have great insights, they have a great team, but then they’re afraid they’ll be roadkill when large companies come in. So we saw this with Zoom and Slack announced their AI products. People are generally afraid of what they don’t know.
And you have been on this journey as well, what were some surprising, maybe positive or negative things, where you saw other forces at play. How do you help founders navigate this? Do you recommend they jump in and figure it out, or figure it out before they jump in? Based on your experience, what’s your advice?
Rodrigo:
I’m not sure what “figure it out” means. I suppose I’d say that if at times if you’re not a little bit afraid, you’re probably not an entrepreneur. You are probably jumping into things that are uncertain by nature because you’re either going into a green field where you don’t know if the market will ever develop or you’re going into a mature market where there’s an incumbent that will most likely come and try to kill you. And so I think you have to have a healthy level of fear as you go into those markets. And yet your conviction shall overcome it all.
Day after day when you’re doing this and things show up, not just you, your team relies on your conviction around this. It’s by nature an unknown path that you’re taking. And so you have to have the conviction to say, yeah, this still makes sense. Or like you said, at times, we have to make a course correction and that’s okay. That’s okay.
But the conviction around changing the world and actually doing something that actually is going to make the world better in a certain way. That comes from us. Comes from founders, comes from the leaders of the company. So I would say that that’s all healthy. I think, like I said before, I think if you are going into every day feeling true assurance that there’s no risk. I’m so certain.
That there are one or two things that’s happening. Either you’re not pushing hard enough or you’re not entirely aware of the situation. But I think if it’s worth doing, it’s always going to come with some level of uncertainty, which should test every day the conviction that you have around the endeavor that you’re pursuing.
Chris:
I guess from my perspective, if you’re saying how do you avoid becoming roadkill? If that’s the question. Just to boil it down, right? My advice is that AI is not enough. When you’re building a company, you’re not just building the AI part of the product, right? You’re building a team. You’re building relationships with customers. You’re building the product more broadly. You’re building a brand. You’re building so many of these other things. And AI becomes a part of the enabling technology that feeds it and leads to a product experience.
And so my advice is always start from the customer, work backwards from their pain points. How is it that you’re making their lives better? And then remember, with Rodrigo’s totally correct advice, you need conviction, but have conviction in the fact that execution matters a lot. It turns out that they’re a lot of really poorly run companies. Particularly the big guys. The big guys that are out there that theoretically can accidentally stomp on you and squish you.
They get tied up in knots with incentive structure problems. They don’t want to disrupt an existing product line. They have an ego tied to the existing product because somebody became the VP of whatever by building your thing that they can’t disrupt. And so there’s all these structural advantages that you can have and you can move much faster because you’re a small and focused company, and you ideally know what winning looks like.
And if you start from that question, how do I make the customer happy? How do I solve their problem? And you work backwards from that to AI, that’s a very different outcome, than if you start from AI and work forward to how can I make this algorithm work in a way that somebody will pay for? I think that’s quite a different approach.
Vijay:
I think that’s really good advice. I think we have time for Q&A.
Audience member:
Thank you. Early stage founder here. Thank you for sharing those stages of stories from the beginning, those are awesome and inspiring. We seem to be living in a world now where there’s this, on the computer question in particular, there are quadrillion parameter models in the future. Do you see that to be the future, or do you think small models are going to dominate? And the second I think is on hallucinations. We hear a lot about that every day. Do you see that as something that will be eventually solved or are you going to be 15 years from now still worrying about this AI that is out of control? Thank you.
Rodrigo:
So we have a point of view on that. Again, companies are going to do whatever makes sense for themselves. And so some companies use what, 180 billion, 250 billion parameter models as a single monolithic or as a large chunk mixture of experts. And so companies will do what makes sense for them. Now, if you’re asking SambaNova for a recommendation, and we talked about this yesterday during our launch, our view of it is what we’re calling composition of experts.
So what that is, is basically thinking about these in terms of small models. So the one that we showed yesterday, we did 150 experts of LLaMA-2-7B. Is to put 150 experts of small models all side by side, all being able to actually be routed through one switch model that allows you to then give you full control of the experts and who gets access to what.
And the reason why enterprises like that is because they can then incrementally add LLaMA-2s. I don’t have to buy into a trillion. The one that was running yesterday was a trillion parameters. We can go up to five, but think about this not as, oh my God, this is a trillion. Think about this as I started with 20 LLaMA-2 experts, and then I just let my team start training, fine tuning more LLaMA-2s and adding them into the portfolio.
And so that’s our point of view. That’s the sum of it. A way that we think that companies can add and create these very, very valuable experts. And again, today, the experts, again, it’s on public data. Imagine how enterprises, I can go into my HR database, I can go into my contracts database, I can go into the Rodrigo Private. You can take individual data sets, train LLaMA-2 models for yourself, and then put security access on that expert.
And so allows you tremendous flexibility. So that’s our view of how we think enterprises will go. But I think these types of things have to support any model that people want to use. But one thing I know is as for enterprises, bigger is better on the accuracy side, but people continue to push the envelope. I never thought it was going to get this big, but bigger seems to be better. And people continued to push the envelope.
Chris:
I think that I’d split the answer into two different things, the research and then the enterprise side. I think research will continue to push large models. I think that we’re going to see 10X bigger models. We’re going to see more new novel model architectures that lead to better accuracy for a given size, sparsity, all these things that can come in and help and mixtures of experts and all these things are all awesome.
I think that on the enterprise side, there’s more of a utilitarian question. And I’m seeing a lot of people that say, okay, I’m told by the internet that I’m supposed to be deploying LLMs or generative or whatever into my products. And they’re looking at the cost. And having a DGX box for every inference query or something like that just doesn’t really stack up.
And so I think that the real question ends up being this R O I question. Because a larger model will give you better results. It’ll have better prediction accuracy and things like this. But is it better enough to be worth paying it for? And this is where suddenly the product value proposition starts to come in. So I think that it can be different from that perspective.
Jules:
Hey, how’s it going? My name is Jules. Earlier career product manager at LinkedIn. And I have a question for you guys. So you kind of spoke about how you spent a lot of time focusing on the customer’s new pain points in your needs. So what issue that I run into a lot is I’m always hopping into solutions. So for you, you guys talk about how much time you spend with the customer? And also I also hear about sometimes the customer doesn’t know what they want. So how did you balance spending time with the customer and also figuring out exactly what they needed?
Chris:
So in our case, I like to say that Modular’s, it’s almost a two year-old company at this point, but it’s based on six or seven years worth of research. And so I’ve been working in the AI space since 2016. And so across that time, I’ve seen a lot of new innovations. I’ve seen the Resonant 50 performance wars, I’ve seen the dawn of BERT and the Transformer, I’ve seen the rise of generative, and all these different things. And so started with a pretty deep background in a lot of the technologies and kind of what works, what doesn’t. What people talk about and why.
And one of those insights was that researchers talk, but enterprises don’t, right? And so if all your data comes from NeRV’s papers or things like this, you get a very different view than what happens if you actually talk to people and see what they’re actually deploying into their infrastructure.
And so when we started the company, we had a big backlog of understanding of how the industry worked. But a lot of it was formed by being at a big company or wearing somebody else’s hat. And so when you’re a startup, I think it’s really good to revalidate your assumptions. And so my co-founder and I, Tim, lead a lot of the product and the business focus side of things. I lead the technology side.
And so when we started the company, I built the engineering team, started building products, or doing all that kind of stuff. He went out and interviewed, I think it was 75 different organizations. They’re all building and deploying AI into their products. And did so with the brand new startup hat on. And then yes, we leveraged relationships and prior reputations and things like this to get a door to open.
But really starting and spending the time on that is incredibly valuable. And a lot of that really shaped our focus. And for example, early on we were debating, do we focus on mobile first or data center first? There’s a lot of good arguments for mobile is where personalized AI goes. Has better latency benefits. Has all these things. You have a supercomputer in your pocket, all of these AI accelerators that almost nobody is using.
But we decided that actually it’s kind of too early for the average enterprise to be thinking about that. And so it’d be better for us to start on the data center side. To give you just one really simple pivot point that we discussed.
Rodrigo:
I think from an enterprise SaaS perspective, we asked four questions. One is, who is the person you’re going after? Do they have budgets? And are they the decider, or is somebody else deciding for them, right? Last but not least, is it a painkiller or is it something which is nice to have? So if you check off all these boxes, then you know you have some ICP initially to go with the customer. And then you can expand and it can go off different groups. But the wedge has to be pretty strong when you’re going after enterprise SaaS products.
Vijay:
Yeah, I would add that you should be intentional about your customer engagements. You can spend a lot of time educating your customers about life.
Rodrigo:
Particularly in AI. Everybody wants help.
Vijay:
And you’re going to be giving free tutorials. And so I would say be intentional. In different phases of the company you’re going for different things. The person that will give you product feedback may not actually be your eventual buyer. They may all be wearing the same badge, but they may not be the buyer. So be intentional about it. This stage of the company, I’m looking for feedback on my product. Let me learn that. This phase, I’m trying to understand where I break into that company. What was the best place to start a collaboration? So every product, every company is going to have a different way to do it, but my suggestion is be intentional about how you engage. Because otherwise, you can spend a lot of time.
Audience member:
Hi, my name is Sab, early stage founder and very insightful. Thank you. My question is around building the team. One is the value of multidisciplinary background in the team, especially talks about ethical AI and all that. And two, you mentioned value is very expensive and culture is very important. So how do you build culture in a virtual team? Thank you.
Chris:
So I think there’s a couple of different questions there. One is how do you identify and recognize talent? The second is, what do you do with it when you find it? In our case, I don’t know what the right answer is. Some people go for the pedigree of your graduate of the best institution or something, right? In our case, we don’t do that. Instead, what we do is we really focus on the, if you’re an engineer, the technical skill sets, can you code?
Do you have the right background? Do you have a GitHub profile or something like that to give us additional data? We do a culture interview for everybody we hire. And that’s a really important part because we’ve all heard about the 10X programmers that nobody could work with and things like this. And so I think that if it’s important, you should spend time on it.
When selecting and recognizing people, it’s actually really difficult because there’s so many people, particularly when you get out of a super-specialized niche. RTL designer for branch protectors kind of a role. You have to be more flexible and look to people that can inspire you somehow. And this is one of the things where we hired some, one engineer had a high frequency trading place.
He had no experience with languages, compilers, or this kind of technology at all, but he had a real passion for performance. And real passion for understanding how things worked. And it turned out to be an amazing hire. And so I don’t know if there’s a right answer, but I think that really starting from what is it that you care about? How do you want to build your company? How do you want people to work together? What is that culture that you want? I think that then you can institute that into the interview process. And then you have to get the funnel of candidates coming in as well, which is a different problem.
Rodrigo:
I agree. I agree with most of what Chris said here. I think you had a second piece of the question around breadth versus depth on the people. I would say those people that can cross disciplines early on are just so valuable to a company. Because it’s a somewhat selfish interest. If they’re not doing that, you’re doing it, right? And so you want to find somebody that can help you figure out, well, how do these things work together and then start making progress.
And one of the things that we struggle with as a company, now we’re 500 people and we’re trying to figure out how to actually create more depth in very specific functions. But those crossover folks are just so incredibly valuable. Because they’ll come in and say, “Well, why are you fixing over here? But that should be fixed over there.” Right?
So I think there’s still that level of glue that you need in the company, but it does come to a point where you just can’t find enough of these global utility. So you have to start building the org with depth in mind.
Vijay:
I think we’re out of time. Thanks for all the questions. Once again, if you can give a round of applause to both Chris and Rodrigo. Thanks so much for being here.
