NVIDIA CEO Jensen Huang

In this episode of Acquired, hosts Ben Gilbert and David Rosenthal celebrate Nvidia's 30th anniversary with its founder and CEO, Jensen Huang. They delve into the company's evolution from graphics to data centers and AI, examining Nvidia's strategic position and pondering its future dominance in the AI market. Huang shares pivotal moments, such as the high-stakes launch of the Reva 128 chip, and discusses the importance of building a computing platform with a consistent architecture, like CUDA, to foster a robust developer ecosystem. He also touches on the acquisition of Mellanox and its significance for Nvidia's data center ambitions. Additionally, Huang addresses the societal implications of AI, emphasizing the need for safety, human-in-the-loop systems, and the potential for AI to create jobs through increased productivity. Throughout, Huang reflects on the challenges of entrepreneurship, the support systems that helped him, and his unwavering commitment to Nvidia, offering insights on company building and the relentless pursuit of opportunity.

• Ben Gilbert and David Rosenthal express appreciation for Nvidia's production team's assistance during recording.
• The convenience of having a professional team handle technical aspects of filming was highlighted.
• Ben jokes about upgrading their home studio with high-quality equipment.

"I will say, David, I would love to have Nvidia's full production team every episode. It was nice not having to worry about turning the cameras on and off and making sure that nothing bad happened myself while we were recording this."

The quote reflects the positive experience of having a professional team handle the recording process, allowing the hosts to focus on content creation.

• Ben Gilbert introduces the "Acquired" podcast, which discusses technology companies and their strategies.
• David Rosenthal is introduced as the co-host.
• The episode features a visit to Nvidia's headquarters and an interview with Jensen, Nvidia's CEO.

"Welcome to this episode of Acquired, the podcast about great technology companies and the stories and playbooks behind them. I'm Ben Gilbert."

This quote serves as the opening line for the podcast, setting the stage for the episode's focus on Nvidia and its CEO, Jensen.

• Nvidia's valuation and strategic position are discussed.
• The hosts ponder the sustainability of Nvidia's dominance in the AI market.
• Jensen's leadership and the company's journey, including near-death experiences, are key discussion points.

"At the time of recording, Nvidia is worth $1.1 trillion and is the 6th most valuable company in the entire world."

The quote emphasizes Nvidia's market position and valuation, underscoring the importance of the company's strategies and future prospects.

• David Rosenthal shares that the interview provided new insights into Nvidia and Jensen as a founder.
• The hosts acknowledge that even with extensive research, direct interaction offers deeper understanding.

"Yeah, I got new perspective on the company and on him as a founder and a leader just from doing this."

The quote reveals the value of firsthand interactions in gaining a more nuanced understanding of a subject.

• The hosts invite listeners to join their Slack community for discussions.
• A disclaimer is made that the podcast is for informational and entertainment purposes and may include personal investments.

"All right, well, listeners, join the slack. There is incredible discussion of everything about this company, AI, the whole ecosystem, and a bunch of other episodes that we've done recently going on in there right now."

This quote encourages listener engagement and directs them to a platform for further discussion, while also providing a standard disclaimer about the nature of the podcast's content.

• Jensen recounts the critical period before shipping the Riva 128 graphics chip.
• The company faced financial constraints and made a risky decision to test entirely in simulation.
• Despite only supporting a fraction of the intended features, Nvidia had to market the Riva 128 effectively.

"You're getting ready to ship the Riva 128, which is one of the largest graphics chips ever created in the history of computing."

The quote sets the context for one of Nvidia's pivotal moments in its early history, highlighting the significance of the Riva 128 chip.

• Jensen explains the strategic pivot Nvidia made with the Reva 128.
• The company had to adapt to DirectX and compete with numerous emerging competitors.
• Nvidia's decision-making at the time was crucial for survival and future success.

"Revo 128 was a reset of our company, because by the time that we realized we had gone down the wrong road, Microsoft had already rolled out DirectX."

The quote captures the moment Nvidia had to reassess its strategy and adapt to the changing technological landscape to stay competitive.

• Nvidia used emulation to prototype the Riva 128 chip due to financial limitations.
• The company's leadership was confident in the chip's perfection because of thorough testing.
• Jensen describes the company's all-in approach, including a marketing blitz, based on their confidence in the product.

"We essentially virtually prototype the chip by buying this emulator."

The quote explains the innovative approach Nvidia took to ensure the Riva 128 chip was thoroughly tested and ready for production, despite financial constraints.

• Jensen discusses the importance of conviction and preparation when making significant company bets.
• He emphasizes the need to simulate and test thoroughly before making a major strategic move.

"When you push your chips in, what you're really doing is when you bet the farm, you're saying, I'm going to take everything in the future, all the risky things, and I pull it in advance."

The quote highlights Jensen's philosophy on making big bets for the company, stressing the importance of pulling future risks into the present and addressing them proactively.

• Jensen reflects on Nvidia's collaboration with researchers and universities.
• He credits these relationships with providing positive feedback and confidence in the potential of deep learning.

"But we had confidence that even Gan, the first time I met Goodfellow, the Gan was like 32 x 32, and it was just a blurry image of a cat, but how far can it go?"

The quote illustrates Nvidia's early belief in the potential of deep learning technology, even when initial results were not yet impressive.

• Jensen discusses the founding of OpenAI and the importance of supporting AI researchers.
• He recounts delivering the first DGX computer to OpenAI and the company's focus on aiding research advancement.

"But most of our success was aligned around in the beginning, just about helping the researchers get to the next level."

This quote conveys Nvidia's commitment to supporting the research community, which was instrumental in advancing AI technology.

• Jensen talks about the impact of deep learning breakthroughs on Nvidia's strategy.
• The company's ability to scale deep learning and its implications for various industries are explored.
• Jensen believes that machine learning could revolutionize software development and computing.

"And if that's the case, then how you build a computer and how you build a chip, in fact, can be completely changed."

The quote reflects Jensen's vision of how deep learning and AI can fundamentally alter the approach to computing and hardware design.

• Ben Gilbert observed the exponential progress of deep learning by monitoring archive papers.
• The frequency of research publication increased, indicating rapid developments in the field.

"So you could just monitor the archive papers. And I took an interest in learning about the progress of deep learning, and to the best of my ability, read these papers, and you could just see the progress happening in real time, exponentially in real time."

The quote highlights Ben Gilbert's method of keeping up with the progress in deep learning by regularly reviewing new research papers, which demonstrated swift advancements.

• David Rosenthal discussed the unexpected effectiveness of large language models as they scaled up.
• Researchers initially believed algorithmic changes were necessary for improvements, but model scaling proved otherwise.
• The crossing of parameter thresholds (10 billion, 100 billion) led to significant enhancements in model performance.

"It even seems like within the industry, from some researchers we spoke with, it seemed like no one predicted how useful language models would become. When you just increase the size of the models, they thought, oh, there has to be some algorithmic change that needs to happen. But once you cross that 10 billion parameter mark, and certainly once you cross the 100 billion, they just magically got much more accurate, much more useful, much more lifelike."

David Rosenthal reflects on the industry's surprise at the effectiveness of scaling up language models without needing algorithmic changes, which led to models becoming more accurate and lifelike.

• Jensen described his first impression of BERT and its clever approach to language modeling through self-supervised learning.
• The process involves masking out words and predicting the next word, which facilitates learning reasoning from text.
• There was an anticipation that BERT would scale up significantly and encode reasoning in its architecture.

"Well, my first feeling about the language model was how clever it was to just mask out words and make it predict the next word. It's self supervised learning at its best."

Jensen expresses admiration for BERT's innovative approach to language modeling, which uses self-supervised learning to predict masked words, demonstrating an intuitive way to teach reasoning to models.

• Speaker E shared a moment of clarity from a college class, understanding the von Neumann computer architecture from programming language to electrical engineering.
• Jensen agreed, noting that the integration of complex systems still feels miraculous even when understood.

"I even remember that from my first operating systems class in college. When I finally figured out all the way from programming language to the electrical engineering classes bridged in the middle by that OS class, I'm like, oh, I think I understand how the von Neumann computer works. Soup to nuts. And it's still a."

Speaker E recalls the moment of comprehending the entire von Neumann architecture, a foundational concept in computer science.

• Ben Gilbert and David Rosenthal discussed the deployment of machine learning models via A/B testing.
• Experimentation tools allowed for gradual rollout and quick feedback, fostering a data-driven culture.
• Statsig was recommended as a platform that offers similar tools to those used by leading companies like Google and Facebook.

"The most common way to deploy new models was through experimentation a b testing. When the research team created a new model product, engineers would deploy the model to a subset of users and measure the impact of the model on core product metrics."

Ben Gilbert explains the importance of A/B testing in deploying new ML models, allowing for measured impact and iterative improvements based on user data.

• Jensen explained that Nvidia's organizational structure is not based on a military hierarchy but is more akin to a computing stack.
• The company is structured to reflect the architecture of the products they build, with each layer managed by the most capable individual.
• The phrase "mission is the boss" encapsulates the company's approach to collaboration and goal achievement.

"Nvidia is not built like a military. It's not built like the armed forces, where you have generals and colonels. We're not set up like that. We're not set up in a command and control and information distribution system."

Jensen describes Nvidia's unique organizational structure, which does not follow a traditional hierarchical model but is designed more like a computing stack with an emphasis on expertise and mission-driven work.

• Jensen discussed the importance of aligning a company's structure with its product architecture.
• He emphasized the need for leaders to anticipate future opportunities and position the company accordingly.
• The leadership style at Nvidia is based on reasoning through problems and enabling team success, rather than on privileged information.

"Your organization should be the architecture of the machinery of building the product. Right. That's what a company is."

Jensen articulates his philosophy on organizational design, emphasizing that a company's structure should mirror the process of creating its products.

• Jensen recounted Nvidia's strategic move into the data center space, which began 17 years ago.
• The separation of computing from viewing devices was identified as a critical factor for expanding market opportunities.
• The development of cloud gaming and remote graphics led to Nvidia's growth in the data center market.

"Wouldn't it be amazing if our computer doesn't have to be connected to the viewing device, that the separation of it made it possible for us to compute somewhere else?"

Jensen discusses the foresight that led to Nvidia's pivot towards data center computing, recognizing the potential of separating the computing process from the physical constraints of traditional desktop setups.

• Jensen highlighted the importance of positioning for future opportunities, even without perfect foresight.
• He drew parallels between leadership strategies and the principles shared by Warren Buffett and Charlie Munger about being approximately right.

"Our job as CEO is to look around corners and anticipate where will opportunities be someday. And even if I'm not exactly sure what and when, how do I position the company to be near it."

Jensen emphasizes the role of a CEO in anticipating future opportunities and strategically positioning the company to capitalize on them when they arise.

• Jensen explained that acquiring Melanox was strategic for Nvidia's expansion into data center networking.
• The acquisition was driven by the need for high-performance networking to enable distributed computing for AI.
• Melanox's expertise in networking was seen as essential for Nvidia's vision of future computing.

"And so the day that I concluded that, really, Nvidia wants to build computers of the future, and computers of the future are going to be data centers embodied in data centers."

Jensen shares his rationale behind the acquisition of Melanox, emphasizing the strategic move to align Nvidia with the future of computing in data centers.

• Crusoe has built an AI cloud solution focused on high-performing GPUs.
• They use Nvidia GPUs to meet the demand for AI workloads.
• Crusoe offers Nvidia's H scale, which is in high demand due to a scarcity in the market.
• Their strategy involves using the best GPU hardware and investing in optimized cloud software.
• Crusoe's data centers run on clean energy, providing better performance per dollar.
• They have reduced emissions significantly by capturing gas and avoiding CO2 emissions.

"Crusoe has been ahead of the curve and is among the first cloud providers to offer Nvidia's H scale."

This quote emphasizes Crusoe's proactive approach in offering Nvidia's H scale, which is a sought-after product in the AI cloud market.

"They can provide significantly better performance per dollar than traditional cloud providers."

This quote highlights Crusoe's competitive advantage in terms of cost efficiency and performance, achieved through the use of clean energy.

"In fact, in 2022, Crusoe captured over 4 billion cubic feet of gas, which led to the avoidance of approximately 500,000 metric tons of CO2 emissions."

This quote underlines Crusoe's positive environmental impact by capturing gas that would otherwise contribute to CO2 emissions.

• The biggest competitor for startups is often the lack of demand for their product.
• As companies grow and find product-market fit, they need to consider competitive strategy.
• Nvidia's approach is to position in markets that are not yet established, termed "zero billion dollar markets."
• Nvidia has historically entered markets early, focusing on future needs like software-defined cars.
• By being early, Nvidia often faces less competition when the market emerges.
• Nvidia has been successful in PC gaming, workstations, supercomputing, and AI through this strategy.

"We prefer to position ourselves in a way that serves a need that usually hasn't emerged."

This quote explains Nvidia's strategy of entering markets before they fully develop, addressing future needs and reducing immediate competition.

"We spent about a decade in zero billion dollar markets."

This quote indicates Nvidia's long-term approach, investing in markets that did not exist yet but believing in their future potential.

• Nvidia is considered a platform company, which is rare.
• Initially, Nvidia was a technology company but always had the vision of being a platform.
• Nvidia's architecture, UDA, and later CuDa, were foundational to becoming a platform.
• Developer relations were crucial from the beginning to ensure support for Nvidia's technology.
• The transition from technology to platform was natural for Nvidia due to its original vision.

"Inside our company, we were always a platform company."

This quote reveals that Nvidia's internal strategy was always to become a platform company, even when it began as a technology company.

"Every Nvidia chip will run CUdA."

This quote demonstrates Nvidia's commitment to ensuring architectural compatibility across all its accelerators, forming the basis of its platform.

• AI safety is paramount, especially in areas like robotics and self-driving cars.
• Nvidia is focused on functional safety and the concept of human-in-the-loop for AI systems.
• AI is expected to create more jobs in the near term due to increased productivity and prosperity.
• Companies that become more productive usually expand and hire more people.
• Nvidia encourages everyone to learn AI to augment their productivity and job security.

"We have to keep AI safe."

This quote stresses the importance of ensuring AI systems are reliable and secure to prevent harm.

"My feeling is that, and we'll see, but it is more likely that AI is going to create more jobs in the near term."

This quote expresses the belief that AI will lead to job creation due to increased productivity and the expansion of industries.

• Starting a successful company is extremely difficult and requires both luck and skill.
• Even with smart decisions, companies can fail; conversely, mistakes can still lead to success.
• Nvidia's early decisions, like the design of Revo 128, were critical to its success.
• Luck played a role, such as John Carmack's decision to use acceleration for Quake.
• Nvidia's philosophy was to build a developer-oriented platform from the start.

"I just think starting successful companies are insanely hard."

This quote acknowledges the inherent difficulty in building a successful company, regardless of the decisions made.

"If Carmack hadn't decided to use acceleration... what would be the killer app that put us here?"

This quote recognizes the role of luck in Nvidia's success, specifically the external decision by John Carmack that favored their technology.

• Nvidia's initial strategy was to be a game console within the PC.
• They prioritized relationships with game developers from the start.
• DirectNv was Nvidia's original architecture, which later aligned with Microsoft's DirectX.
• Nvidia learned from early challenges with developer platforms and evolved their strategy accordingly.

"Direct Nvidia, yeah. And DirectX was an API that made it possible for operating system to directly address hardware."

This quote explains Nvidia's original plan to create a direct interface for hardware, which later had to adapt to the prevailing DirectX standard.

"We were always a developer oriented company."

This quote underscores Nvidia's consistent focus on cultivating strong relationships with developers, which was foundational to its strategy.

• Technology increases efficiency, leading to more hiring and expansion in companies.
• Human ambition is described as never-ending, driving continual expansion and energy consumption.
• Every version of our species has always attempted to pursue more ideas over time.

"The world has infinite ambition. If a company is more profitable, they tend to hire more people to do more. That's true."

The quote emphasizes the notion that profitability and success in business lead to expansion and the hiring of more employees due to the boundless nature of human ambition.

"Humans will always expand and consume more energy and attempt to pursue more ideas. That has always been true of every version of our species over time."

This quote underlines the historical consistency of human ambition to expand and innovate, suggesting a fundamental trait of human nature.

• Blinkist provides summaries of books, including a collection related to tech innovation, leadership, and acquisitions.
• The service offers key insights from books, curated reading lists, and progress tracking for personal and team development.
• An exclusive discount is provided to listeners for Blinkist's premium content.

"Blinkist has handpicked a collection of books related to the themes of this episode. So tech innovation, leadership, the dynamics of acquisitions, these books offer the mental models to adapt to a rapidly changing technology environment."

This quote highlights the relevance of Blinkist's curated book collection to the themes discussed in the podcast, emphasizing the importance of adapting to technological changes.

"Blinkist is giving acquired listeners an exclusive 50% discount on all premium content. This gives you key insights from thousands of books at your fingertips, all condensed into easy to digest summaries."

The quote informs listeners of the promotional offer from Blinkist, emphasizing the accessibility and convenience of the service for gaining insights from a wide range of books.

• Jensen shares his favorite science fiction series and a personal story involving his favorite car.
• The story includes a memorable event with his fiancée and the significance of the car to him.
• Jensen's daily driver is now a Mercedes EQS, chosen for security reasons and its excellent quality.

"Star Trek's my favorite."

Jensen's preference for the "Star Trek" series is stated, indicating his interest in science fiction television.

"I'm driven in the Mercedes EQs. It's a great car."

This quote shares Jensen's current choice of vehicle, the Mercedes EQs, and his positive opinion of it.

• Jensen appreciates the intuitiveness and approachability of Clay Christensen's series on business.
• He also values the insights from Andy Grove's books.
• Don Valentine's characteristic of being "grumpy but endearing" is fondly remembered, along with a humorous anecdote.

"I think the series is the best. I mean, there's just no two ways about it. And the reason for that is because it's so intuitive and so sensible."

Jensen praises Clay Christensen's series for its sensible and intuitive approach to business concepts.

"What he said to me the last time as he decided to invest in our company. He says, if you lose my money, I'll kill you."

This quote reflects the candid and impactful interaction Jensen had with investor Don Valentine, highlighting Valentine's direct and humorous way of expressing his investment expectations.

• Jensen believes there is plenty of time if one prioritizes properly and does not let external schedules dictate their time.
• He emphasizes the importance of prioritizing life, making sacrifices, and controlling one's own schedule.

"There's plenty of time if you prioritize yourself properly and, and you make sure that you, you don't let outlook be the controller of your time."

Jensen discusses the concept of time management, stressing the importance of self-prioritization over being controlled by a schedule or external demands.

• Jensen's primary fear is letting down the employees who have invested their belief in the company's vision.
• He feels a strong responsibility to ensure success for his employees, allowing them to build great lives and careers.

"The greatest fear is that you let them down."

This quote encapsulates Jensen's concern for the well-being and success of his employees, which he considers to be his greatest fear as a CEO.

• Jensen reflects on his career path and the influence of LSI Logic on his philosophy about computing.
• He discusses the impact of high-level design on productivity and the revolutionary changes it brought to the industry.

"If it wasn't because of Chris and Curtis convincing me to do Nvidia, I would still be at LSI logic today."

Jensen acknowledges the significant turning point in his career when he decided to co-found Nvidia, which was influenced by his colleagues Chris and Curtis.

"What LSI logic did was realized that you can express transistors and logical gates and chip functionality in high level languages."

This quote discusses the innovation at LSI Logic that allowed for a higher level of abstraction in chip design, which Jensen found revolutionary and compelling.

• Jensen admits that building a company is much harder than anticipated and involves significant challenges.
• He believes entrepreneurs often underestimate the difficulty of starting a company, which can be a "superpower" in driving them forward.
• Jensen also discusses the potential for technology companies to grow substantially by focusing on the manufacturing of intelligence and work.

"Building a company and building a video turned out to have been a million times harder than I expected it to be, any of us expected it to be."

This quote reflects on the unexpected difficulties encountered in building Nvidia, highlighting the underestimation of the challenges involved.

"Don't be surprised if technology companies become much larger in the future because what you produce is something very and that's the kind of the way to think about how large can your opportunity, how large can you be?"

Jensen speculates on the future growth of technology companies based on the immense market opportunities associated with the manufacturing of intelligence and work-related technology.

• Jensen emphasizes the importance of having a support system of family, friends, and colleagues to endure the emotional challenges of building a company.
• He expresses gratitude for the unwavering support he received from those around him throughout Nvidia's journey.

"You need the unwavering support of people around you."

This quote stresses the necessity of having a strong support system to succeed in the emotionally taxing process of building and running a company.

• The hosts encourage listeners to sign up for their email list, join their Slack community, and participate in the selection of future podcast episodes.
• They also mention the availability of merchandise and the opportunity to engage more deeply with the hosts through their LP program.

"So if you want to play the little guessing game along with the rest of the acquired community, sign up at acquired FM Slash email."

The hosts invite listeners to join their email list for updates and to engage with the community by guessing future episode topics.

"Anyone should join the slack. Acquired FM Slash Slack God, we've got a lot of things now, David."

This quote encourages listeners to join the Acquired Slack community to discuss episodes and engage with other members.