Nvidia Part III The Dawn of the AI Era (20222023)

In this episode, Ben Gilbert and David Rosenthal of Acquired dive into the remarkable growth and transformation of Nvidia, particularly over the last 18 months, which have witnessed an explosion in the AI sector. They explore Nvidia's strategic pivot from a GPU manufacturer to a full-stack data center and AI platform company, focusing on the release of their H100 GPUs and the integration of generative AI into various applications. The discussion highlights Nvidia's significant investments in hardware, software (notably CUDA), and networking (through the acquisition of Mellanox), positioning them as a leading force in the AI revolution. They also consider the implications of Nvidia's growth on the broader tech industry, including cloud providers and the potential for a shift in data center spending towards Nvidia's solutions. The episode concludes with a reflection on Nvidia's competitive moat, the scalability of AI workloads, and the company's culture of innovation under the leadership of Jensen Huang.

• Ben Gilbert and David Rosenthal exchange pleasantries, discussing Ben's visit to Bucks and the nostalgia there.
• They consider inviting Jensen Huang from Nvidia to Bucks and adding Nvidia memorabilia to the establishment.
• They transition into the main content of the podcast.

"I went for the first time, what, two weeks ago when I was down for meeting at benchmark. And the nostalgia in there is just unbelievable."

This quote highlights Ben's recent visit to Bucks and the strong sense of nostalgia he experienced there, setting a friendly and casual tone for the podcast.

"I can't believe you hadn't been before. I know Jensen is a Denny's guy, but I feel like he would meet us at Bucks if we asked him."

David expresses surprise at Ben's first-time visit to Bucks and humorously suggests that Jensen Huang might join them there despite his preference for Denny's.

• The podcast hosts discuss the rapid changes in AI technology, specifically mentioning the absence of the word "generative" in their previous episodes due to the pace of change.
• They recount the economic downturn of 2022 and its impact on the tech industry, including Nvidia's inventory write-off.
• The breakthrough of large language models (LLMs) is highlighted, with Chat GPT's significant user growth and the comparison to Netscape and iPhone moments.
• They emphasize Nvidia's foundational role in the hardware and software that powers these AI advancements.

"In our April 2022 episodes, we never once said the word generative. That is how fast things have changed."

This quote underscores the swift evolution of AI technology and the introduction of generative models into mainstream discussion, reflecting the theme of rapid change in AI.

"By the fall of 2022, right when everything looked the absolute bleakest, a breakthrough technology finally became useful."

This quote describes the timing of the AI breakthrough during an economic downturn, highlighting the sudden and impactful emergence of practical AI applications.

• The hosts delve into the history of AI, starting with the AlexNet moment in 2012 and its significance in machine learning.
• They discuss the use of neural networks and Nvidia's GPUs in accelerating AI research.
• The podcast explores the acquisition of AI talent by Google and Facebook and the resulting duopoly in AI research.
• They introduce OpenAI's founding and its mission to democratize AI research beyond the control of big tech companies.

"AlexNet was three researchers from the University of Toronto who submitted the AlexNet algorithm to the ImageNet computer science competition."

This quote provides historical context for the AlexNet breakthrough, which propelled AI research forward by demonstrating the effectiveness of neural networks in image recognition.

"So thousands of people on Mechanical Turk got paid, however much, $2 an hour to label these images."

This quote explains the labor-intensive process of labeling images for the ImageNet dataset, which was essential for training early AI models before the advent of more sophisticated techniques.

• Elon Musk and Sam Altman are introduced as the figures behind the founding of OpenAI, with a dinner at the Rosewood hotel leading to the recruitment of Ilya Sutskever.
• The podcast discusses the challenges of AI research monopolization by big tech and the vision for OpenAI to create artificial general intelligence (AGI) in an open manner.

"I felt there were risks involved, but I also felt it would be a very interesting thing to try."

This quote from Ilya Sutskever captures his willingness to leave Google and join OpenAI, despite the risks, to pursue the goal of open AI research.

"I think there are three levels of concern here. One obviously is the other tech companies. Then there's the problem of startups. This is terrible for startups."

This quote reflects on the wider implications of AI research concentration within Google and Facebook, including the impact on competition and innovation in the tech industry.

• The concept of large language models (LLMs) is introduced, explaining their ability to predict the next word in a sentence based on a large context window.
• The 2017 Google paper "Attention Is All You Need" and the transformer model are discussed, highlighting their role in enabling parallel processing of sequence-based models.
• The podcast touches on the computational cost of training LLMs and the exponential increase in parameters from GPT-1 to GPT-4.

"One algorithm I'm excited about is a language model. The idea that you can take a large amount of data and you feed it into the network, and it figures out the pattern in how words follow each other in sentences."

This quote from Andrej Karpathy, then at OpenAI, foreshadows the development of chatbots and LLMs that learn language patterns from vast amounts of internet data.

"The attention mechanism is o of n squared."

This quote explains the computational complexity of the attention mechanism in transformers, which, despite being resource-intensive, can be effectively parallelized with GPUs.

• Google provided free cloud computing resources to OpenAI in its early days.
• The availability of Google's resources was a strategic move to increase the user base of Google's infrastructure.
• Google Cloud's offering allowed scale for AI compute.

"But this was three, four years early and a very prescient move to really get a lot of people using Google architecture. Compute at scale."

The quote emphasizes the foresight Google had in offering their cloud resources, understanding the future importance of AI and the need for scalable compute power.

• OpenAI faced the reality of high costs to become a cutting-edge AI company.
• Elon Musk's frustration led to his departure from OpenAI, which was a pivotal moment in the company's history.
• OpenAI pivoted towards transformer models, which required significant investment in compute power.

"Elon gets super frustrated by all this. Basically throws a hissy fit and quits and pieces out of OpenAI."

This quote highlights Elon Musk's dissatisfaction with the direction of OpenAI, leading to his exit and marking a significant turning point for the company.

• OpenAI decided to create a for-profit entity to raise capital for AI model development.
• The company capped profits for investors to align with their mission.
• Microsoft invested $1 billion in OpenAI and became the exclusive cloud provider for the organization.

"On March 11, 2019, OpenAI announced it was creating a for-profit entity so it could raise enough money to pay for all the compute power necessary to pursue the most ambitious AI models."

The quote explains OpenAI's strategic shift to a for-profit model to secure the necessary funding for advanced AI research and development.

• OpenAI released GPT-3, and Microsoft licensed its commercial use exclusively.
• Microsoft made additional investments in OpenAI, leading to the development of GitHub Copilot and ChatGPT.
• Microsoft's integration of GPT into its products and further investment highlighted the importance of cloud-based GPU compute.

"Microsoft invests another $10 billion in OpenAI, announces they're integrating GPT into all of their products."

This quote signifies the deepening partnership between Microsoft and OpenAI, with substantial financial backing and strategic integration of AI technologies into Microsoft's product suite.

• Nvidia spent years developing a GPU-accelerated computing platform for data centers.
• The advent of generative AI and the need for GPU compute created a perfect opportunity for Nvidia.
• Nvidia's investment in networking company Melanox and the development of the Grace CPU processor positioned them well for the AI era.

"Nvidia has literally just spent the past five years working insanely hard to build a new computing platform for the data center."

This quote underscores Nvidia's long-term vision and effort to revolutionize data center computing, which aligned perfectly with the rise of generative AI.

• Nvidia's acquisition of Melanox provided high-speed data interconnects for data centers.
• The Grace CPU and Hopper GPU architectures were developed for AI workloads.
• Nvidia's integrated solutions and proprietary technology catered to the demands of generative AI.

"Nvidia makes one of the best acquisitions of all time back in 2020, and nobody had any idea. They bought a quirky little networking company out of Israel called Melanox."

The quote highlights Nvidia's strategic acquisition of Melanox, which played a crucial role in enhancing data center capabilities, particularly for AI applications.

• AI models require large amounts of high-bandwidth memory to be effective.
• Nvidia's use of chip on wafer on substrate (CoWoS) technology allowed for more memory on GPU chips.
• The memory constraints necessitated networking multiple chips and servers to handle AI training.

"The constraint today is actually in how much high performance memory is available on the chip."

This quote points out the critical role of on-chip memory in AI computing, emphasizing the challenges and innovations required to meet the demands of AI model training.

• Nvidia's DGX systems offer a turnkey AI data center solution for enterprises.
• DGX Cloud provides a virtualized DGX system for companies preferring cloud-based solutions.
• Nvidia's direct sales relationship with enterprises through DGX Cloud strengthens their market position.

"Nvidia has now introduced DGX Cloud, which is a virtualized DGX system that is provided to you right now via other clouds."

The quote introduces Nvidia's DGX Cloud service, which allows customers to access powerful AI computing resources virtually, without the need for physical infrastructure.

• Nvidia's revenue growth is driven by the demand for generative AI compute.
• The company's strategic positioning in the AI market has led to significant financial success.
• Nvidia's focus on data center solutions and AI compute has paid off with increased revenue forecasts.

"Nvidia forecasts Q2 revenue of $11 billion, which would be up another 53% quarter over quarter over Q1 and 65% year over year."

The quote conveys Nvidia's remarkable financial growth, fueled by the burgeoning demand for AI computing power, illustrating the company's successful navigation of the AI landscape.

• Nvidia, once valued at $800 billion, experienced a 25% increase in share price post-earnings.
• In April, Nvidia was the 8th largest company globally, with a market cap of approximately $660 billion.
• The company's market cap plummeted below $300 billion but surged back to over $1 trillion within months.
• These fluctuations underscore the volatility and rapid changes in the tech industry's valuation landscape.

"Back when we did our episodes last April, Nvidia was the 8th largest company in the world by market cap, had about a $660,000,000,000 market cap that was down slightly off the highs, but that was kind of the order of magnitude back then. It crashed down below 300 billion, and then within a matter of months, it's now back up over a trillion."

This quote highlights Nvidia's dramatic shift in market capitalization, reflecting investor sentiment and market conditions that can rapidly alter a company's valuation.

• Nvidia's Q2 fiscal '24 earnings were considered one of the most incredible by any scaled public company.
• The earnings release signified a historic moment due to the substantial revenue growth and market impact.
• The significance of Nvidia's earnings is emphasized as an event that will be remembered regardless of future outcomes.

"This was a historic event. I think this was one of, if not the most incredible earnings release by any scaled public company ever."

The quote emphasizes the unprecedented nature of Nvidia's earnings release, marking it as a historical milestone in the company's and the tech industry's financial history.

• Nvidia's data center segment alone generated $10 billion in revenue for the quarter, more than doubling the previous quarter's revenue.
• The growth signifies the delivery of products to customers, not just preorders or speculative metrics.
• The data center segment's revenue accounted for a significant portion of the total company revenue, highlighting its importance to Nvidia's business.

"Their data center segment alone did $10 billion in the quarter. That's more than doubling off of the previous quarter."

This quote underscores the significant revenue growth within Nvidia's data center segment, which is a concrete indicator of the company's performance and delivery of products to customers.

• Nvidia frames its trillion-dollar opportunity as centered around the data center market.
• The company identifies $1 trillion worth of hard assets in data centers worldwide, with an annual spend of $250 billion on updates and additions.
• Nvidia positions itself as the most cohesive, fulsome, and coherent platform for the future of data centers, indicating a strategic focus on this area.

"There is $1 trillion worth of hard assets sitting in data centers around the world right now, growing at $250,000,000,000 a year."

The quote outlines the vast economic potential Nvidia sees in the data center market, with a trillion-dollar valuation of existing assets and significant annual spending on data center capital expenditures.

• Nvidia's growth is partly attributed to the real user value created by AI workloads and applications.
• Evidence of this value is seen in the success of OpenAI and the widespread adoption of GPT-like experiences.
• Nvidia's strategy involves anticipating that every application will eventually have a GPT front end, transforming human-computer interaction.

"The thing you have to believe is there is real user value being created by these AI workloads and the applications that they are creating."

This quote reflects the belief that AI workloads are generating tangible value for users, which is a critical factor in Nvidia's growth and the adoption of its technology.

• Blinkist condenses books into key points, allowing users to read or listen to summaries.
• Nvidia and Blinkist collaborate to create a curated bookshelf and a special collection related to AI and Nvidia's history.
• The collaboration offers users access to knowledge and insights that have influenced Nvidia's leadership.

"Blinkist takes books and condenses them into the most important points so you can read or listen to the summaries."

This quote describes the service provided by Blinkist, which aligns with Nvidia's initiative to educate users and provide access to influential content.

• CUDA, initiated in 2006, is Nvidia's bet on scientific computing beyond graphics.
• It has become the foundation for all AI applications, with a robust developer ecosystem.
• CUDA's flexibility and support have contributed to Nvidia's competitive edge, with millions of registered developers and a growing community.

"CuDA has become the foundation that everything that we've talked about, all the AI applications are written on top of today."

The quote highlights CUDA's central role in Nvidia's strategy and its significance as the underlying foundation for AI applications and developer engagement.

• Nvidia is perceived as a platform company, akin to Microsoft, with a comprehensive operating system and programming environment.
• The company's approach to data center solutions and its developer ecosystem positions it as more than just a hardware provider.
• Nvidia's focus on full-stack development while accommodating market needs showcases its strategic flexibility.

"Nvidia thinks of themselves as, and I believe is a platform company."

This quote conveys the perception of Nvidia as a platform company with a broad scope, similar to industry giants like Microsoft, underscoring its strategic positioning in the market.

• Scale economies and network economies create a unique brand of power.
• Nvidia is recognized for its branding power, akin to IBM's reputation in the past.
• Nvidia's market-leading products in AI and graphics provide brand power, especially in corporate decision-making.

"This is the nobody gets fired for buying IBM. I mean, Nvidia is the modern IBM in the AI era."

This quote draws a parallel between Nvidia's current reputation in AI and IBM's historical reputation for reliability, indicating that Nvidia is a safe and reputable choice for corporate technology purchases.

• Nvidia successfully transitioned its consumer brand recognition to enterprise.
• Their technology leadership and the "magical" capabilities of their products have been well-known for decades.
• A "strength leads to strength" phenomenon where Nvidia's revenue results bolster its brand benefit.

"They carried their consumer brand into their enterprise posture."

Nvidia's strong consumer brand has positively influenced its enterprise reputation, suggesting that the brand's strength and technological leadership have seamlessly transferred from consumer to corporate markets.

• Nvidia's process power is considered the weakest among its advantages.
• Their culture and shipping cycle have been influential, with a historical six-month shipping cycle.
• Competitors may struggle to match Nvidia's rapid development pace.

"I think you can make an argument here. Is it feasible? Let's do a thought exercise. Could any of their competitors really, in any domain, move to a six month ship cycle? That'd be really hard."

The quote challenges the feasibility of competitors matching Nvidia's rapid shipping cycle, implying that Nvidia's process power, while not the strongest, still presents a significant competitive barrier.

• Jensen Huang, Nvidia's CEO, refers to a new mainstream method for interacting with computers as the "iPhone moment" for AI.
• Nvidia's parallel to Apple's success with a vertically integrated hardware and software stack.
• Nvidia's B2B target market is less price-sensitive and values high-quality experiences, similar to Apple's strategy with the iPhone.

"It's quite tongue in cheek to be referring to the iPhone moment of AI when referring to oneself, Nvidia, as the Apple..."

This quote discusses Nvidia's self-comparison to Apple during a transformative period in AI, highlighting the strategic and market similarities between Nvidia's approach to AI and Apple's approach to the iPhone.

• Nvidia has evolved from a hardware company to a systems company.
• The company's focus is on how well multiple GPUs work together, not just individual chip performance.
• Nvidia's philosophy is to build a great company by doing things others can't, rather than competing on common ground.

"You build a great company by doing things that other people can't do."

This quote encapsulates Nvidia's strategic philosophy of focusing on unique, high-value innovations rather than engaging in commoditized competition, which is reflected in their approach to product development and market positioning.

• Nvidia's competitive drive has been partly fueled by a desire to challenge Intel's dominance.
• Nvidia's patient strategy has allowed them to innovate and expand their product offerings at the right time.
• The company's investment in breakthrough innovations has positioned them well in large markets.

"You only get to do the stuff they're doing if you invested ten years ahead of the industry."

The quote suggests that Nvidia's current success is the result of a long-term, forward-thinking investment strategy that allowed them to innovate ahead of the industry and capture significant market opportunities.

• The incumbency of AWS and other cloud providers may or may not translate to dominance in the cloud AI era.
• Nvidia's full-stack experience may challenge the existing cloud providers' moat.
• The future of cloud services may not be a settled frontier, with potential new vectors for competition.

"Will winning cloud 1.0 for all these Google, Microsoft, Amazon... Will that toe hold actually enable them to win in the cloud AI era?"

The quote raises the question of whether the current dominance of major cloud providers will extend into the AI-driven future of cloud computing, suggesting that Nvidia's specialized AI offerings could disrupt the existing cloud landscape.

• The bull case includes Nvidia's potential to capture a significant share of the growing AI and accelerated computing market.
• The bear case considers the possibility of an overhyped AI market, potential competition from big tech companies, and shifts in computing workloads.
• Nvidia's unique position and rapid innovation pace may allow it to navigate through potential market fluctuations.

"Yeah, I mean, never doubt big tech's ability to throw tens of billions of dollars into something if the payoff could be big enough."

This quote reflects the bear case concern that large tech companies may heavily invest in competing with Nvidia if the AI market proves to be lucrative, posing a threat to Nvidia's dominance.

• Nvidia's competitive moat is nearly impossible to replicate due to its advanced GPU chips, networking capabilities, and software ecosystem.
• Competing with Nvidia would require surpassing their continuous innovation and convincing developers and customers to switch from established Nvidia products.
• The company's entrenched position makes it a formidable force in the AI and accelerated computing market.

"So I think the bottom line here is nearly impossible to compete with them head on."

The quote summarizes the difficulty any competitor would face in trying to directly challenge Nvidia's dominance, emphasizing the company's comprehensive and well-entrenched market position.