20Product How Linkedin Does Product Reviews, A PostMortem on Stories, Linkedin Messenger and Spam & Why the Data Advantage in AI is Diminishing with Tomer Cohen, CPO @ Linkedin

In a comprehensive discussion on "20 Product" hosted by Harry Stebings, Toma Cohen, CPO at LinkedIn, delves into the evolving landscape of AI, particularly the diminishing edge of data exclusivity due to technologies like GPT, which are trained on vast public datasets. Toma underscores the significance of AI in product development, emphasizing the need for product leaders to adeptly navigate AI tools. He reflects on his journey to LinkedIn, highlighting pivotal product strategies and the importance of a growth mindset. Toma also discusses the challenges of product innovation, the balance between art and science in product management, and the potential of AI to revolutionize industries beyond tech. The conversation touches on the impact of AI on content publishing and data privacy, with Toma advocating for responsible AI development and the pursuit of new knowledge generation.

• Data was previously considered crucial in AI, but its importance is waning.
• Technologies like GPT are trained on vast amounts of public data.
• The concept of pretraining on public information is a significant inflection point.
• The traditional data advantage is decreasing as AI becomes more sophisticated.

"If you ask me, a couple of years ago, I would have told you that data is everything in AI. What's happening right now is technologies like GPT are already trained on all public data. The whole idea of the pretrained is one of the biggest inflection points of this technology. It's already trained on every available public information out there. The data advantage that used to exist, I think, is getting diminished."

This quote emphasizes the shift in the AI landscape, where the abundance of data for training models like GPT has reduced the competitive edge that data once provided.

• Toma Cohen is the CPO at LinkedIn, having joined in 2012.
• His background includes entrepreneurship and working with Greylock Partners.
• Toma has been involved in launching and scaling new experiences for LinkedIn members and customers.
• His passion for building and problem-solving led him to LinkedIn, inspired by Reid Hoffman's vision.

"I came to the valley in 2008. I went to a lecture at a Stanford engineering school. It was about social networks... On stage, there was more of an older founder. His name was Reid Hoffman... Reed talked about the power of online professional communities and how it can create economic opportunity... And it was only several years later I had a conversation with who back then was the CPO of LinkedIn... And he said, instead of talking about it, how about you come and build it? And the rest is history. I joined the company 2012, and in 2020 I became the CPO myself."

Toma Cohen recounts his journey to becoming CPO at LinkedIn, highlighting his early inspiration from Reid Hoffman and his progression within the company.

• Toma believes that product development intertwines art and science.
• Science in product development includes best practices and applied learning.
• Art involves vision, creativity, intuition, and the ability to anticipate needs and industry trends.
• Successful product people combine knowledge with creativity, from vision to execution.

"I think it's impossible to delineate science from art. I think they're ruven, they play off each other... But I think what sets you apart in the craft of building product is your ability to bring vision and creativity and intuition and the judgment and the imagination... And I think the further you grow in your career, there is an expectation that's what you'll bring to the company."

Toma Cohen discusses the balance of art and science in product development, emphasizing the importance of creativity and vision in distinguishing oneself in the field.

• Toma acknowledges the need for improvement in LinkedIn's UI.
• LinkedIn is working on innovating within the constraints of its current experience.
• The platform's use case has evolved, and there's potential to simplify the experience with AI.
• LinkedIn has seen significant growth in revenue, user base, and engagement.

"I think there's a lot that we can do there to improve, and there's that actually, we're working on right now to really innovate within the constraints of what we've done... In fact, the use case of LinkedIn dramatically evolved over the last few years."

This quote reflects Toma's recognition of the need to update LinkedIn's user interface to better serve its evolving use cases and leverage AI for simplification.

• Being wrong but not confused is preferable as it implies clarity and focus.
• An example is LinkedIn's launch of Stories, which did not meet expectations.
• LinkedIn learned that users wanted permanence, not ephemerality, in their content.
• This insight led to a focus on creation as part of one's professional identity on LinkedIn.

"We launched stories a while back... We thought ephemerality might alleviate that concern, and we launched it... When we did follow up sessions with members after that, it was clear that we completely misunderstood the job to be done."

Toma Cohen shares a learning experience with LinkedIn Stories, where user feedback revealed a misunderstanding of user needs, prompting a strategic pivot.

• Being first to market is less important than achieving product market fit.
• First to product market fit gains a significant advantage in insights, momentum, and speed.
• LinkedIn's approach to Stories was not about competing with Snap but about addressing specific user needs on their platform.

"I don't think it's first to launch. I think first to launch is not the right concept... I think it's first to product market fit. That is amazing. If you're first to product market fit, you build an amazing leg up in terms of both insights and momentum and speed."

Toma Cohen discusses the importance of being first to achieve product market fit over simply being first to launch a product, as it leads to a stronger position in the market.

• Establish clear definitions of success before product launch.
• Success is indicated by real adoption and retention, not just initial user engagement.
• Product decisions should start with a strong conviction and hypotheses, then seek data to validate them.
• Incremental validation and investment in a product or feature is key.
• Conviction can drive initial decisions when data is insufficient.

"Yeah, I always like to start with before you launch, what is success once you launch it? What should be going up into the right that should be excited about?"

This quote highlights the importance of setting specific success metrics before launching a product to objectively measure its performance.

"When you have real adoption and retention, that's when you know you really have something."

Adoption and retention are the ultimate indicators of a product's success, as they reflect genuine user interest and value.

"It's really a gradual incremental process that starts with conviction when you don't have enough data, but ideally you come back with data to showcase the responses again."

This quote emphasizes the process of starting with conviction-based decisions and then using data to iteratively validate and refine the product.

• Regular product reviews are conducted weekly.
• Reviews cover major investment areas and set expectations for product teams.
• The goal is to improve products through diverse and deep feedback.
• The agenda is set by the product leader, focusing on problem definitions, insights, and principles.
• Product reviews are a collaborative effort involving multiple perspectives from various business areas.

"Yeah, so we do multiple product reviews every week."

Regular product reviews are a staple activity to ensure continuous improvement and alignment with business goals.

"Our goal as a team in this meeting is to make that thinking better, is to make that product better."

The purpose of product reviews is to collectively enhance the product and the team's approach to solving problems.

"By the time the meeting is done, they're getting some pretty diverse, deep feedback from multiple areas of the product, and that allows for that deep thinking to come to it."

Feedback from various stakeholders contributes to comprehensive understanding and refinement of the product.

• In-person product discussions are preferred for their dynamic energy and creativity.
• Remote discussions are viable but considered less effective.
• Post-COVID, in-person sessions resumed to capitalize on the benefits of face-to-face interaction.

"I feel there is an energy of creativity and discussion that you get in the room in person."

In-person interactions foster a more vibrant and creative atmosphere, which is beneficial for product development.

"The energy levels are just higher. The creativity, the ideation, the velocity of discussion is elevated to a whole new level."

The quote suggests that in-person meetings enhance the quality and pace of discussion, leading to better outcomes.

• The product leader sets the agenda for product reviews.
• Sessions are structured, beginning with problem definition and ending with a demo.
• Principles guide the approach to problem-solving, including trade-offs between different objectives.

"I set the agenda for the quarter in terms of what I would like to see as the product areas being covered."

The product leader has the responsibility of guiding the focus of product reviews to align with strategic objectives.

"Principles are basically your opinionated approach for how you would solve the problem."

Principles serve as the framework for how product teams approach problem-solving, balancing different factors and trade-offs.

• Feedback from product reviews is summarized and prioritized.
• The presenting team is responsible for acting on feedback and setting timelines for key areas.
• A "briefback" process ensures clarity and accountability in execution.

"I try to make sure it's between one to three, so it's not the whole list."

Prioritizing a small number of key areas ensures focus and manageability in addressing feedback.

"It's really up to the team presenting to take that feedback and act on it."

The team that presents in the product review is responsible for incorporating feedback into their action plans.

• Changing the LinkedIn feed to focus on member-driven content was controversial.
• The decision involved a trade-off between discovery and user experience.
• High conviction and evidence were required to support the change.

"The first change I made there was that the feed was first and foremost about people that matter to you, talking about things you care about."

This quote explains the fundamental shift in the LinkedIn feed's purpose, prioritizing member content over promotional content.

"There was high conviction there, and I had to show evidence along the way."

The decision to revamp the LinkedIn feed was driven by a strong belief in its necessity, backed by evidence gathered over time.

• AI is transforming product development and leadership.
• Product leaders must be skilled in AI to guide their teams effectively.
• AI requires a shift from deterministic control to guiding principles and refining learning algorithms.

"Those pedals, for me, are AI. And that guide better be you as the product leader."

AI is likened to a critical tool for navigating product success, emphasizing the role of the product leader in mastering it.

"With AI, you don't control the experience. AI is not deterministic."

This quote captures the fundamental change in product management with AI, where outcomes are influenced by machine learning rather than direct control.

• AI tools like GitHub Copilot are increasing productivity.
• The fast pace of AI development is unprecedented.
• AI is setting new baselines for innovation and productivity.

"I've been using this technology all the time. It's incredible."

The speaker endorses AI tools like GitHub Copilot for their impact on productivity.

"If you go with all the constraints of AI today... those will all be alleviated."

The speaker anticipates that current limitations in AI will be overcome, leading to even greater advancements and productivity gains.

• Incumbents are currently innovating effectively with AI, as seen with companies like Microsoft and Adobe.
• Startups have a window of opportunity due to their ingenuity and ability to innovate from scratch.
• Incumbents possess advantages such as resources, market share, and proprietary data.
• Startups can rethink problems and innovate across various industries.
• The balance of power between incumbents and startups in the AI space is dynamic.

"The really interesting thing now is I think incumbents are innovating so well with AI. You look at Microsoft, you look at Adobe and how Adobe are integrating into core products and product suite."

The quote highlights the current trend of established companies successfully incorporating AI into their products, illustrating the competitive edge they hold in the market.

• The previous belief was that data was paramount in AI, with data being likened to oil.
• The emergence of technologies like GPT, which are pretrained on public data, is reducing the data advantage.
• Unique applications and fine-tuning with proprietary data can create specialized tools.
• Fine-tuning is an expensive process requiring significant computing power.
• Startups have the opportunity to innovate with AI by leveraging their unique data sets.

"If you ask me a couple of years ago, I would have told you that data is everything in AI because computing power is accessible and the models are accessible to old or open source. So it's really about the data you have."

This quote explains the historical importance of data in AI and how it used to be the main differentiator in developing AI applications.

• The rise of AI necessitates new skill sets within teams.
• AI talent is becoming increasingly important in all companies.
• Mastery of prompts and communication with AI is a critical skill.
• The best practices for AI are still being established, requiring teams to adapt and learn.

"I think you need new skill sets in your team, for sure. You need people who are working very closely with this technology."

This quote emphasizes the need for teams to evolve and incorporate new skills to work effectively with AI technologies.

• The role of AI in publications and content discovery is evolving.
• AI could change the way users interact with content, potentially bypassing traditional platforms.
• The attribution of sources by AI tools like Bing's copilot is important for retaining value for content platforms.
• The potential changes in internet business models and information discovery due to AI are still unfolding.

"I think that will potentially start changing the interaction model."

This quote suggests that AI technologies may lead to significant changes in how users interact with and access content online.

• The power of AI brings both excitement and concern.
• Responsible AI principles such as transparency, inclusivity, and privacy are crucial.
• The use of AI as a tool or a weapon depends on the responsibility of the builders.
• The question of AI's alignment with political correctness depends on the intended use and application.

"The notion of being extremely responsible with how this technology is being used, especially for the ones building it, I could not agree with more."

This quote underscores the importance of ethical considerations and responsible development in the AI industry.

• Current AI models are focused on existing public knowledge.
• The potential of AI to hypothesize and create new knowledge poses questions about the future of scientific discovery.
• AI's ability to generate new knowledge represents a new frontier for business, society, and human-machine interaction.
• The societal adaptation to the implications of advanced AI, such as AGI, is still to come.

"Imagine AI coming up with answers to some of the biggest scientific mysteries in the world, like what is dark matter? What's dark energy, what causes Alzheimer's disease, what is quantum mechanics, what is oneself?"

This quote reflects the profound impact AI could have on our understanding of the world by potentially solving complex scientific questions.

• Discussion of the potential of self-driving technology extending beyond cars to areas like medicine and psychology.
• The idea that technology could intimately understand individuals through access to personal data, with permission.
• Mention of the transformative impact of such technology on society and its resemblance to science fiction becoming reality.

"What about a self driving doctor, a self driving psychologist? Imagine this technology being able to be so intimate with you. It could access your phone, your computer, your photo, obviously, with your permissions, it could read your voicemail, every piece of digital footprint."

• The quote reflects on the vast potential applications of self-driving technology in intimate roles like healthcare and personal assistance, emphasizing the importance of user permission for data access.

• Concerns about the relevance of current education in the face of rapid technological change.
• The concept of a 'decay rate' of education, questioning the usefulness of what children learn today for their future adult life.

"Tommy, you have children. Do you feel a bit ridiculous sending them off to school every day?"

• This quote introduces the topic of the relevance of current educational content in preparing children for the future workforce.

• Toma Cohen emphasizes the importance of a growth mindset over specific skills for future success.
• The rapid pace of technological change demands continuous learning and adaptability.
• Growth mindset is considered a fundamental value in Toma Cohen's household.

"The only skill that matters, I think, is growth mindset."

• Toma Cohen identifies growth mindset as the most critical skill, highlighting its importance over any specific knowledge or skill set due to the fast-evolving nature of technology and job requirements.

• Discussion on the current phase of technology being focused on foundational models.
• Foundational models serve as a base for building more specialized and tailored applications.
• Expectation of more foundational models emerging in the future, with a current trend towards diversification of models.

"We are now in a phase of foundational models."

• Toma Cohen describes the current technological landscape as being centered around core, versatile models that provide a platform for further innovation and specialization.

• Toma Cohen shares his favorite interview questions for product roles.
• Questions are designed to assess problem-solving skills and attitude towards failure and learning.
• The goal is to identify candidates with a deep understanding of complex problems and a growth mindset.

"One is, what's the most complex problem you worked on and how did you do it?"

• This quote reveals one of Toma Cohen's preferred interview questions, aiming to understand a candidate's approach to complex problem-solving and their ability to articulate it.

• Toma Cohen discusses his pride in successful product releases, especially those with significant impact, such as LinkedIn's transformation into a mobile company.
• He does not feel embarrassment for less successful releases, viewing them as learning opportunities.
• An example of a challenging project is the attempt to launch instant articles on LinkedIn, which faced timing and market adoption issues.

"Transforming LinkedIn into a mobile company was an incredible moment to rethink."

• Toma Cohen reflects on a proud moment in his career, highlighting the significant shift in strategy and the successful transition of LinkedIn to a mobile-first platform.

• Toma Cohen advises new product leaders to understand the broader ecosystem of the company they join.
• He encourages thinking beyond one's specific area to create unique and innovative experiences that leverage the company's entire ecosystem.

"Your ability to be exponentially successful at LinkedIn is really learning how LinkedIn works."

• The quote advises new product managers at LinkedIn to grasp the complex ecosystem of the company to drive innovation and success beyond their immediate responsibilities.

• Toma Cohen shares his experience with intermittent fasting and its benefits for both physical and mental capabilities.
• He suggests giving the fasting routine a few weeks to adjust and discusses his personal fasting schedule.

"I'm sharper when doing my fast than when I eat, so it's actually elevated my physical abilities and my mental capabilities."

• Toma Cohen describes the positive effects of intermittent fasting on his cognitive and physical performance, advocating for its potential benefits on health and longevity.

• Toma Cohen expresses admiration for Microsoft's integration with OpenAI.
• He highlights the strategy, execution, and impact of this collaboration.
• Microsoft's role as a partial owner of OpenAI is clarified, with OpenAI remaining an independent company.

"I've been very impressed with what Satya and Microsoft has done with the OpenAI integration."

• The quote conveys Toma Cohen's high regard for Microsoft's strategic partnership with OpenAI, which he views as a groundbreaking and responsibly executed product strategy.

• Toma Cohen and the host conclude the interview with mutual appreciation for the discussion.
• The host acknowledges Toma Cohen's resilience to the questions and thanks him for his participation.

"This was awesome. Thank you so much."

• Toma Cohen expresses his enjoyment of the interview, indicating a positive and engaging conversation with the host.