In this episode of 20vc, Harry Stebbings interviews Richard Socher, a pioneering AI researcher and the founder and CEO of U.com, discussing the impact of AI on job automation and the economy. Socher, formerly the chief scientist at Salesforce and CEO of Metamind, highlights the uneven distribution of AI's benefits and the challenge of jobs without digital data to automate. He also addresses the complexities of AI development, the importance of retrieval augmentation for language models, and the potential overestimation of AGI's near-future impact. Additionally, the conversation touches on the role of AI in society, the balance between innovation and distribution, and the future of U.com as a search platform. Promotional segments interspersed throughout the podcast feature Coda's AI-powered work assistant, Navan's travel credit rewards, and Public.com's treasury accounts.
"The future is already here, it's just not equally distributed and I think it will meaningfully change so many jobs. The more data there is about a job, the more that job is likely to be automated."
This quote highlights the disparity in the adoption of technology across different jobs and sectors, emphasizing the role of data availability in automation.
"What that means is that the tasks that are physical are getting more and more expensive, and they're going to become the new bottleneck, and then they're going to slow down overall progress so that the GDP can't like 100 x because of AI."
The speaker is explaining that physical tasks, which are harder to automate, are becoming costlier and could limit the overall economic growth expected from AI advancements.
"I'm so excited to welcome Richard Socher, founder and CEO of U.com."
Harry Stebbings introduces Richard Socher, highlighting his achievements and contributions to the AI field.
"Well, Coda is here with their new AI-powered work assistant that helps you and your team not just finish tasks, but make progress."
Harry Stebbings discusses the benefits of using Coda's AI-powered work assistant for improving team productivity and task management.
"Navan rewards your employees with personal travel credit every time they save their company money when booking business travel under company policy."
The speaker promotes Navan's unique approach to incentivizing cost-effective travel booking by rewarding employees, suggesting potential cost savings for companies.
"As of the 29 June, you can get 5.5% yield on your cash with 26 week treasury bills."
The speaker informs listeners about the opportunity to earn a high yield on cash using Public.com's service for purchasing Treasury bills.
"Boy, that goes back to 2003 is when I started linguistic computer science at Leipzig University."
Richard Socher gives a brief history of his journey into the ML and NLP field, dating back to his university days.
"We are at the beginning of an exponential improvement in a lot of different capabilities. At the same time, some people think the exponential will just keep on going."
Richard Socher discusses the current state of AI, noting both rapid advancements and the tendency for people to overestimate the continuous pace of breakthroughs.
"So there is an existence proof for a single model for all of NLP."
Richard Socher argues for the feasibility of a single model capable of handling all NLP tasks, drawing an analogy to the human brain's versatility.
"You just cannot train a single model for all of these different tasks with a small model."
Richard Socher emphasizes the importance of model size in creating a versatile, single model for various NLP tasks.
"You need to learn everything about the geography of the northeastern United States. And now if you do this billions of times across the Internet, on all the chemistry articles and biology articles and so on, you learn world knowledge."
The quote emphasizes the scope and scale of learning required for a language model to develop a comprehensive understanding of world knowledge, using geography as a specific example.
"But there are still a lot of data sets out there that are not out there. They're actually stored in private databases."
This quote highlights the dichotomy between publicly available data and the vast amounts of data stored in private databases, which can be a competitive advantage for incumbents.
"So there's a lot. And then there are also areas where the default large language model will not do as well."
The quote suggests that while foundational models are powerful, the real value and differentiation come from additional layers and components that companies build on top of these models.
"But hallucinations aren't a feature and are a problem for search engine."
This quote acknowledges that while creative deviations can be entertaining, accuracy is crucial when it comes to providing factual information in a search engine context.
"We're going to update all these models like we update our model every other week."
The quote indicates the rapid pace at which language models are updated, reflecting the dynamic nature of AI development.
"It won't matter that much which lm you use, but it matters what you do with it, how you tune it, what kind of training data you add onto it to fine tune it."
This quote suggests that the strategic use and customization of language models are more critical than the choice of a specific model.
"I predicted that we'll have a GBD four equivalent model before the end of the year. That's open source."
The quote reflects an expectation that open-source models will continue to improve and become more accessible, challenging the dominance of proprietary models.
"Due on the show from contextual said that actually it was the lack of alignment between open ecosystems which would lead to closed winning."
The quote presents an argument that the organizational structure and focus of closed ecosystems might give them an edge over open ecosystems in the AI industry.
"But that would be my dream, is that we actually have a single model. And just like Wikipedia, the whole world collaborates and adds to it."
This quote underscores the aspiration for a collaborative AI model akin to Wikipedia's open editing system, emphasizing the potential for global enhancement and accessibility.
"The problem is it's very hard for governments to fund uncertain research and science projects because when it doesn't work out, some taxpayers will for sure complain."
This quote highlights the difficulty governments face in funding research with uncertain outcomes due to the risk of public backlash and the need for bureaucratic oversight.
"But more importantly, intuitively, we ask search engines questions every day to learn something."
This quote emphasizes the daily relevance and significant impact of search engines on learning, suggesting the importance of advancements in this area of NLP technology.
"We need to have that search engine that we've built be an open platform where you can actually contribute your apps to and your content to."
This quote suggests a vision for future search engines, where content providers can directly contribute and potentially profit from their contributions, fostering a more collaborative ecosystem.
"The truth is distribution won't be ever fully solved and it's a constant uphill battle."
This quote acknowledges the persistent challenge startups face in achieving distribution and the need for continuous innovation and strategic partnerships to compete with established players.
"I think it'll be a mix."
This quote suggests that both startups and established companies will derive value from advancements in AI, with neither group having a definitive advantage.
"But progress is not as linear or as exponential as a lot of people think."
This quote cautions against assuming that technological progress will continue at a consistent rate, using historical examples to illustrate that advancements can slow or plateau.
"You think, like, we have this keep having exponential growth. But I'll give you an example where we already see the flattening out of that exponential growth into an s curve, and that is an image generation."
This quote points out that the perceived exponential growth in AI may be overstated, with some areas like image generation already approaching their limits, signaling a possible plateau in progress.
It doesn't make sense to have superhuman language because humans couldn't understand it anymore to some degree.
The quote emphasizes that AI advancements in language must remain comprehensible to humans, as surpassing human understanding would render the AI's abilities ineffective for practical use.
It's weird that we've reached plateauing technological development with zero awareness of it in 6.9 billion people.
Harry Stebbings points out the odd situation where technological advancements have reached a high level, yet the vast majority of the global population remains unaware of these developments.
The future is already here. It's just not equally distributed.
This quote, cited by the speaker, illustrates that while the future with all its advancements exists, it is not accessible or known to everyone equally.
The more digital they are, the more data there is about a job, the more that job is likely to going to be automated and improved massively in its efficiency.
The speaker suggests that jobs with more digital presence and data availability are more likely to be automated, improving their efficiency significantly.
Are you concerned by the job displacement question and just the awareness that when you look at industrial revolutions and technological revolutions, there's always decade long, actually transition periods, whereas here it seems like the transition period is years.
Harry Stebbings is asking about the concern regarding the rapid pace of job displacement due to AI compared to slower transitions in past industrial revolutions.
And I feel empathy with those people. And it would be nice to have social systems that catch some of those people and help them learn new kinds of skills, help them incorporate those new technologies into their workflows.
The speaker expresses empathy for those affected by AI's impact on jobs and advocates for social systems that could help these individuals adapt and learn new skills.
We do have a lot of research breakthroughs we still need to make in order to achieve AGI.
The speaker acknowledges that significant research progress is required to reach AGI, indicating that it is a complex and ongoing challenge.
Knowledge is similar to the future is already here, but not equally distributed. How important retrieval augmentation is for llms. A lot of people underestimate that.
The speaker believes that the significance of retrieval augmentation in the context of large language models is underestimated by many people, mirroring the uneven distribution of future advancements.
Well, Coda is here with their new aipowered work assistant that helps you and your team not just finish tasks, but make progress so your product team can bring a feature to market faster.
This promotion highlights Coda's AI-powered work assistant, which is designed to help teams be more efficient and productive in their tasks.