In episode 26 of Recsperts, I speak with Sanne Vrijenhoek, a PhD candidate at the University of Amsterdam’s Institute for Information Law and the AI, Media & Democracy Lab. Sanne’s research explores diversity in recommender systems, particularly in the news domain, and its connection to democratic values and goals.

We dive into four of her papers, which focus on how diversity is conceptualized in news recommender systems. Sanne introduces us to five rank-aware divergence metrics for measuring normative diversity and explains why diversity evaluation shouldn’t be approached blindly—first, we need to clarify the underlying values. She also presents a normative framework for these metrics, linking them to different democratic theory perspectives. Beyond evaluation, we discuss how to optimize diversity in recommender systems and reflect on missed opportunities—such as the RecSys Challenge 2024, which could have gone beyond accuracy-chasing. Sanne also shares her recommendations for improving the challenge by incorporating objectives such as diversity.

During our conversation, Sanne shares insights on effectively communicating recommender systems research to non-technical audiences. To wrap up, we explore ideas for fostering a more diverse RecSys research community, integrating perspectives from multiple disciplines.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.
Don't forget to follow the podcast and please leave a review

• (00:00) - Introduction
• (03:24) - About Sanne Vrijenhoek
• (14:49) - What Does Diversity in RecSys Mean?
• (26:32) - Assessing Diversity in News Recommendations
• (34:54) - Rank-Aware Divergence Metrics to Measure Normative Diversity
• (01:01:37) - RecSys Challenge 2024 - Recommendations for the Recommenders
• (01:11:23) - RecSys Workshops - NORMalize and AltRecSys
• (01:15:39) - On the Different Conceptualizations of Diversity in RecSys
• (01:28:38) - Closing Remarks

• Sanne Vrijenhoek on LinkedIn
• Informfully
• MIND: MIcrosoft News Dataset
• RecSys Challenge 2024
• NORMalize 2023: The First Workshop on the Normative Design and Evaluation of Recommender Systems
• NORMalize 2024: The Second Workshop on the Normative Design and Evaluation of Recommender Systems
• AltRecSys 2024: The AltRecSys Workshop on Alternative, Unexpected, and Critical Ideas in Recommendation

Papers:

• Vrijenhoek et al. (2021): Recommenders with a Mission: Assessing Diversity in News Recommendations
• Vrijenhoek et al. (2022): RADio – Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations
• Heitz et al. (2024): Recommendations for the Recommenders: Reflections on Prioritizing Diversity in the RecSys Challenge
• Vrijenhoek et al. (2024): Diversity of What? On the Different Conceptualizations of Diversity in Recommender Systems
• Helberger (2019): On the Democratic Role of News Recommenders
• Steck (2018): Calibrated Recommendations

General Links:

• Follow me on LinkedIn
• Follow me on X
• Send me your comments, questions and suggestions to marcel.kurovski@gmail.com
• Recsperts Website

In episode 25, we talk about the upcoming ACM Conference on Recommender Systems 2024 (RecSys) and welcome a former guest to geek about the conference.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.
Don't forget to follow the podcast and please leave a review

• (00:00) - Introduction
• (01:56) - Overview RecSys 2024
• (07:01) - Contribution Stats
• (09:37) - Interview

• RecSys 2024 Conference Website

Papers:

• RecSys '24: Proceedings of the 18th ACM Conference on Recommender Systems

General Links:

• Follow me on LinkedIn
• Follow me on X
• Send me your comments, questions and suggestions to marcel.kurovski@gmail.com
• Recsperts Website

In episode 24 of Recsperts, I sit down with Amey Dharwadker, Machine Learning Engineering Manager at Facebook, to dive into the complexities of large-scale video recommendations. Amey, who leads the Video Recommendations Quality Ranking team at Facebook, sheds light on the intricate challenges of delivering personalized video feeds at scale. Our conversation covers content understanding, user interaction data, real-time signals, exploration, and evaluation techniques.

We kick off the episode by reflecting on the inaugural VideoRecSys workshop at RecSys 2023, setting the stage for a deeper discussion on Facebook’s approach to video recommendations. Amey walks us through the critical challenges they face, such as gathering reliable user feedback signals to avoid pitfalls like watchbait. With a vast and ever-growing corpus of billions of videos—millions of which are added each month—the cold start problem looms large. We explore how content understanding, user feedback aggregation, and exploration techniques help address this issue. Amey explains how engagement metrics like watch time, comments, and reactions are used to rank content, ensuring users receive meaningful and diverse video feeds.

A key highlight of the conversation is the importance of real-time personalization in fast-paced environments, such as short-form video platforms, where user preferences change quickly. Amey also emphasizes the value of cross-domain data in enriching user profiles and improving recommendations.

Towards the end, Amey shares his insights on leadership in machine learning teams, pointing out the characteristics of a great ML team.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.
Don't forget to follow the podcast and please leave a review

• (00:00) - Introduction
• (02:32) - About Amey Dharwadker
• (08:39) - Video Recommendation Use Cases on Facebook
• (16:18) - Recommendation Teams and Collaboration
• (25:04) - Challenges of Video Recommendations
• (31:07) - Video Content Understanding and Metadata
• (33:18) - Multi-Stage RecSys and Models
• (42:42) - Goals and Objectives
• (49:04) - User Behavior Signals
• (59:38) - Evaluation
• (01:06:33) - Cross-Domain User Representation
• (01:08:49) - Leadership and What Makes a Great Recommendation Team
• (01:13:01) - Closing Remarks

• Amey Dharwadker on LinkedIn
• Amey's Website
• RecSys Challenge 2021
• VideoRecSys Workshop 2023
• VideoRecSys + LargeRecSys 2024

Papers:

• Mahajan et al. (2023): CAViaR: Context Aware Video Recommendations
• Mahajan et al. (2023): PIE: Personalized Interest Exploration for Large-Scale Recommender Systems
• Raul et al. (2023): CAM2: Conformity-Aware Multi-Task Ranking Model for Large-Scale Recommender Systems
• Zhai et al. (2024): Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations
• Saket et al. (2023): Formulating Video Watch Success Signals for Recommendations on Short Video Platforms
• Wang et al. (2022): Surrogate for Long-Term User Experience in Recommender Systems
• Su et al. (2024): Long-Term Value of Exploration: Measurements, Findings and Algorithms

General Links:

• Follow me on LinkedIn
• Follow me on X
• Send me your comments, questions and suggestions to marcel.kurovski@gmail.com
• Recsperts Website