In a deep dive with Harry Stebbings on 20VC, Noam Shazir, AI expert and founder of Character AI, shares insights into the evolution and impact of AI technology. Shazir, with his extensive experience from Google Brain to his current venture, discusses the computational challenges and costs of training sophisticated AI models, emphasizing the importance of both model size and training duration. He highlights Character AI's mission to provide a versatile, direct-to-consumer AI platform, contrasting this approach with other companies' B2B strategies. Shazir also touches on the societal implications of AI, including its potential to augment human connection and address global issues. The conversation also delves into the balance between open and closed AI ecosystems, the rapid pace of technological progress, and the unpredictable future of AI applications.
"The size of the model is the bigger challenge. Actually, the number one thing that's important is how much computation you do to train it."
This quote emphasizes that the key factor in training AI models is not just the size but the computational resources expended during training.
"This is 20 vc with me, Harry Stebings, and welcome back to part two of this very special feature week featuring two of the hottest AI companies today."
Harry Stebings sets the stage for the conversation, highlighting the significance of the AI companies being discussed.
"With coder AI, you can reimagine your to do list and how you collaborate so you're not only finishing tasks, but really making progress."
This quote promotes Coda's AI assistant, suggesting it can transform task management and collaboration.
"Yeah, my first project, we were just looking at, why are people not happy using Google and spelling correction was, like, the number one issue."
Noam Shazir identifies user experience with the spelling corrector as a key issue he addressed at Google.
"Google's been just an incredible company. It's brought so much value to billions of people."
This quote reflects Noam Shazir's admiration for Google's impact and the lessons he learned about reaching a wide audience.
"So I'm really inspired by the Google model of full stack end to end, all the way from basic research to launch a product directly to consumers."
This quote expresses Noam Shazir's strategy of emulating Google's end-to-end product development to engage consumers directly.
"But then the other thing is just to push technology forward. There are so many technological problems in the world that could be solved."
The quote highlights Noam Shazir's belief in the broad potential of AI to address a range of critical issues.
"I think our place is to provide useful tools to everyone on earth, to leave people in control."
This quote encapsulates the philosophy behind Character AI's mission to distribute powerful AI tools without seeking to dictate their use.## Superpower of Technology
"I like this sort of motto of a billion users inventing a billion use cases, because that's sort of the superpower of this technology, and it puts our company in the right place."
Noam Shazir articulates the company's philosophy of empowering users to discover diverse applications for their technology, illustrating the transformative potential of user creativity and autonomy.
"Well, one is that we launched that's definitely been a frustration in the past. Things seem potentially too much brand risk at larger companies to actually launch and get it out there."
Noam Shazir highlights the importance of launching products despite potential brand risks, which has contributed to the company's growth by meeting the widespread need for communication and connection.
"A lot of the people who don't have friends and who are not as well connected. One big source of that is just social anxiety."
Noam Shazir explains that the technology serves to assist individuals with social anxiety by providing a practice environment, potentially leading to improved real-life social interactions.
"The main things we need to do make it very general so we're not like cutting down on the use cases."
Noam Shazir discusses the core product challenge of maintaining generality to cater to a wide range of use cases while ensuring the product remains user-friendly.
"The new way of doing things with neural language models has none of that. Like, I could know zero about language in particular, other than it's like a sequence of words."
Noam Shazir describes the shift from complex rule-based systems to neural language models, emphasizing the latter's simplicity and potential despite the user's lack of linguistic knowledge.
"I'd say it's both. I think there's been a lot of technological progress, both quantitatively and qualitatively, in that the models that were there in 2016 were too dumb to be fun."
Noam Shazir acknowledges the dual factors of technological advancements in AI and the evolving perception and excitement of investors and society regarding AI's capabilities.
"Yeah, probably the size of the model is the bigger challenge. We can get a lot of data, but actually, the number one thing that's important is how much computation you do to train it."
Noam Shazir points out that while data is abundant, the computational effort involved in training large models is the most significant challenge in advancing AI technology.## Constraints on AI Models
Computation. So the model we're serving now, we trained last summer and spent about $2 million worth of compute cycles doing it.
The quote explains that computation is the main limiting factor for AI development, highlighting the high financial cost of training sophisticated models.
Both are useful. Like data that you get from users is great because it tells people what users like or what users like in some particular application.
This quote emphasizes the importance and utility of both types of data in the context of AI training and development.
My experience coming from Google is that a startup can move way faster than a big company and can launch products in ways that large companies are just going to move too slow because they're worried about compromising their existing products.
The quote reflects the speaker's belief in the agility and innovation potential of startups compared to larger, more established companies.
There is just so, so much value about to be created, that there's going to be room for multiple players in there.
This quote suggests a future where the AI market is not zero-sum and can support a diverse range of successful entities.
The ability to mess around with things at a small scale is going to lead to way more research being published, even if some of the larger entities are no longer publishing research because they're trying to maintain competitive advantages.
The speaker highlights the importance of open research and experimentation in driving AI innovation, despite the tendency of large companies to keep research private for competitive reasons.
I think the one message I have is that the best applications just haven't even been invented yet, that we're still at like invention of electricity kind of moment, or invention of the computer where we don't really know what the coolest things are going to be.
This quote conveys a desire to shift societal views towards recognizing the early stage and future potential of AI applications.
If these models are hallucinating, which they certainly are, and we advertise that they are, then the use cases that emerge first will be ones for which hallucination is a feature.
This quote describes the strategic approach to AI model hallucinations, framing them as a feature that can drive creative and beneficial use cases.
What is electricity for? Is it for fun? Is it for productivity? We believe that individuals should make that decision.
The quote draws an analogy between AI and fundamental utilities like electricity, suggesting that the purpose of AI should be as versatile and user-determined as electricity's applications.
I don't judge what I do by how much fun it is. It's more like what's the most useful? So very, very happy to be doing what I can.
The quote reflects the speaker's prioritization of utility and contribution over personal enjoyment in their professional role.
I decided to take a change of attitude from what is fun right now to I should be thankful for having the opportunity to do something important and meaningful.
This quote captures the speaker's personal evolution towards valuing meaningful contributions and the influence of parenthood on this change.
There is no direct quote provided for this theme, but it can be inferred from the context that the speaker would encourage their past self to embrace the meaningful aspects of life's challenges, such as parenthood.## Importance of Self-Care and Prioritization
"Not everything in the world is your responsibility, but you should understand what is your responsibility and what isn't your responsibility."
This quote emphasizes the importance of discernment in understanding what one should take responsibility for, which can lead to a more balanced life.
"I think children are the most fantastically interesting catalyst in one's life because it's the most significant change you will ever have in a day."
This quote reflects the speaker's view on the profound and instantaneous impact that having a child can have on an individual's life.
"This technology is just going to get way smarter."
The speaker predicts substantial growth and intelligence in AI technology, indicating a transformative period ahead.
"I think things are going to move very fast."
This quote signifies the speaker's belief in the imminent and rapid progression of AI technology adoption.
"I think externally it looks like entertainment app, but really we are a full stack company."
This quote clarifies the misconception about the company's focus, highlighting its core commitment to AI and product excellence.
"No one knows exactly what is going to work."
The speaker points out the experimental and uncertain nature of AI research, emphasizing the need for proven results to guide the community.
"I was like, okay, you must be able to do something better and more efficient by building a sparse network. That was so wrong..."
This quote illustrates the speaker's journey from a misconception to a deeper understanding of AI's reliance on hardware capabilities.
"I have absolutely no idea. Like, we will see what technology is like then, but it's just important for us to be agile."
This quote expresses the speaker's uncertainty about the distant future and the importance of adaptability in the face of technological evolution.
"I think this has been unlike any interview you've done for you. I feel like the questions have stretched boundaries of parenthood that people didn't ask you before."
The interviewer acknowledges the unique nature of the conversation, which touched on personal aspects not typically discussed in professional interviews.
"I want to say a huge thank you to Nom."
This quote is an expression of appreciation from the host to the guest for contributing to a compelling and diverse discussion.