Recsperts - Recommender Systems Experts

In episode 29 of Recsperts, I welcome Craig Macdonald, Professor of Information Retrieval at the University of Glasgow, and Aleksandr “Sasha” Petrov, PhD researcher and former applied scientist at Amazon. Together, we dive deep into sequential recommender systems and the growing role of transformer models such as SASRec and BERT4Rec. Our conversation begins with their influential replicability study of BERT4Rec, which revealed inconsistencies in reported results and highlighted the importance of training objectives over architecture tweaks. From there, Craig and Sasha guide us through their award-winning research on making transformers for sequential recommendation with large corpora both more effective and more efficient. We discuss how recency sampling (RSS) reduces training times dramatically, and how gSASRec overcomes the problem of overconfidence in models trained with negative sampling. By generalizing the sigmoid function (gBCE), they were able to reconcile cross-entropy–based optimization results with negative sampling, matching the effectiveness of softmax approaches while keeping training scalable for large corpora. We also explore RecJPQ, their recent work on joint product quantization for item embeddings. This approach makes transformer-based sequential recommenders substantially faster at inference and far more memory-efficient for embeddings—while sometimes even improving effectiveness thanks to regularization effects. Towards the end, Craig and Sasha share their perspective on generative approaches like GPTRec, the promises and limits of large language models in recommendation, and what challenges remain for the future of sequential recommender systems. Enjoy this enriching episode of RECSPERTS – Recommender Systems Experts. Don’t forget to follow the podcast and please leave a review. * (00:00) - Introduction * (04:09) - About Craig Macdonald * (04:46) - About Sasha Petrov * (13:48) - Tutorial on Transformers for Sequential Recommendations * (19:24) - SASRec vs. BERT4Rec * (21:25) - Replicability Study of BERT4Rec for Sequential Recommendation * (32:52) - Training Sequential RecSys using Recency Sampling * (40:01) - gSASRec for Reducing Overconfidence by Negative Sampling * (01:00:51) - RecJPQ: Training Large-Catalogue Sequential Recommenders * (01:21:37) - Generative Sequential Recommendation with GPTRec * (01:29:12) - Further Challenges and Closing Remarks Links from the Episode: * Craig Macdonald on LinkedIn [https://www.linkedin.com/in/craigmacdonald/] * Sasha Petrov on LinkedIn [https://www.linkedin.com/in/asash/] * Sasha's Website [https://asash.github.io/] * Tutorial: Transformers for Sequential Recommendation (ECIR 2024) [https://github.com/asash/transformers-for-recsys-tutorial] * Tutorial Recording from ACM European Summer School in Bari (2024) [https://www.youtube.com/watch?v=H732E7qzbis&ab_channel=ACMEuropeanSummerSchoolinRecSys] * Talk: Neural Recommender Systems (European Summer School in Information Retrieval 2024) [https://www.youtube.com/watch?v=Tan2o8osnW8&ab_channel=MohammadA] Papers: * Kang et al. (2018): Self-Attentive Sequential Recommendation [https://arxiv.org/abs/1808.09781] * Sun et al. (2019): BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer [https://arxiv.org/pdf/1904.06690] * Petrov et al. (2022): A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation [https://eprints.gla.ac.uk/275645/2/275645.pdf] * Petrov et al. (2022): Effective and Efficient Training for Sequential Recommendation using Recency Sampling [https://dl.acm.org/doi/10.1145/3523227.3546785] * Petrov et al. (2024): RSS: Effective and Efficient Training for Sequential Recommendation Using Recency Sampling (extended version) [https://dl.acm.org/doi/10.1145/3604436] * Petrov et al. (2023): gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling [https://eprints.gla.ac.uk/301348/3/301348.pdf] * Petrov et al. (2025): Improving Effectiveness by Reducing Overconfidence in Large Catalogue Sequential Recommendation with gBCE loss [https://dl.acm.org/doi/full/10.1145/3699521] * Petrov et al. (2024): RecJPQ: Training Large-Catalogue Sequential Recommenders [https://dl.acm.org/doi/10.1145/3616855.3635821] * Petrov et al. (2024): Efficient Inference of Sub-Item Id-based Sequential Recommendation Models with Millions of Items [https://dl.acm.org/doi/10.1145/3640457.3688168] * Rajput et al. (2023): Recommender Systems with Generative Retrieval [https://papers.neurips.cc/paper_files/paper/2023/file/20dcab0f14046a5c6b02b61da9f13229-Paper-Conference.pdf] * Petrov et al. (2023): Generative Sequential Recommendation with GPTRec [https://arxiv.org/abs/2306.11114] * Petrov et al. (2024): Aligning GPTRec with Beyond-Accuracy Goals with Reinforcement Learning [https://eprints.gla.ac.uk/322171/1/322171.pdf] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/] Disclaimer: Craig holds concurrent appointments as a Professor of Information Retrieval at University of Glasgow and as an Amazon Scholar. This podcast describes work performed at the University of Glasgow and is not associated with Amazon.

In episode 28 of Recsperts, I sit down with Robin Burke, professor of information science at the University of Colorado Boulder and a leading expert with over 30 years of experience in recommender systems. Together, we explore multistakeholder recommender systems, fairness, transparency, and the role of recommender systems in the age of evolving generative AI. We begin by tracing the origins of recommender systems, traditionally built around user-centric models. However, Robin challenges this perspective, arguing that all recommender systems are inherently multistakeholder—serving not just consumers as the recipients of recommendations, but also content providers, platform operators, and other key players with partially competing interests. He explains why the common “Recommended for You” label is, at best, an oversimplification and how greater transparency is needed to show how stakeholder interests are balanced. Our conversation also delves into practical approaches for handling multiple objectives, including reranking strategies versus integrated optimization. While embedding multistakeholder concerns directly into models may be ideal, reranking offers a more flexible and efficient alternative, reducing the need for frequent retraining. Towards the end of our discussion, we explore post-userism and the impact of generative AI on recommendation systems. With AI-generated content on the rise, Robin raises a critical concern: if recommendation systems remain overly user-centric, generative content could marginalize human creators, diminishing their revenue streams.   Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts. Don't forget to follow the podcast and please leave a review * (00:00) - Introduction * (03:24) - About Robin Burke and First Recommender Systems * (26:07) - From Fairness and Advertising to Multistakeholder RecSys * (34:10) - Multistakeholder RecSys Terminology * (40:16) - Multistakeholder vs. Multiobjective * (42:43) - Reciprocal and Value-Aware RecSys * (59:14) - Objective Integration vs. Reranking * (01:06:31) - Social Choice for Recommendations under Fairness * (01:17:40) - Post-Userist Recommender Systems * (01:26:34) - Further Challenges and Closing Remarks Links from the Episode: * Robin Burke on LinkedIn [https://www.linkedin.com/in/robin-burke-b878303] * Robin's Website [https://www.that-recsys-lab.net/home/people/burke] * That Recommender Systems Lab [https://www.that-recsys-lab.net/] * Reference to Broder's Keynote on Computational Advertising and Recommender Systems from RecSys 2008 [https://recsys.acm.org/recsys08/keynotes/] * Multistakeholder Recommender Systems (from Recommender Systems Handbook), chapter by Himan Abdollahpouri & Robin Burke [https://link.springer.com/chapter/10.1007/978-1-0716-2197-4_17] * POPROX: The Platform for OPen Recommendation and Online eXperimentation [https://poprox.ai/] * AltRecSys 2024 (Workshop at RecSys 2024) [https://altrecsys.github.io/] Papers: * Burke et al. (1996): Knowledge-Based Navigation of Complex Information Spaces [https://cdn.aaai.org/AAAI/1996/AAAI96-069.pdf] * Burke (2002): Hybrid Recommender Systems: Survey and Experiments [https://link.springer.com/article/10.1023/A:1021240730564] * Resnick et al. (1997): Recommender Systems [https://dl.acm.org/doi/10.1145/245108.245121] * Goldberg et al. (1992): Using collaborative filtering to weave an information tapestry [https://dl.acm.org/doi/10.1145/138859.138867] * Linden et al. (2003): Amazon.com Recommendations - Item-to-Item Collaborative Filtering [https://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf] * Aird et al. (2024): Social Choice for Heterogeneous Fairness in Recommendation [https://dl.acm.org/doi/10.1145/3640457.3691706] * Aird et al. (2024): Dynamic Fairness-aware Recommendation Through Multi-agent Social Choice [https://dl.acm.org/doi/full/10.1145/3690653] * Burke et al. (2024): Post-Userist Recommender Systems : A Manifesto [https://arxiv.org/abs/2410.11870] * Baumer et al. (2017): Post-userism [https://dl.acm.org/doi/10.1145/3025453.3025740] * Burke et al. (2024): Conducting Recommender Systems User Studies Using POPROX [https://dl.acm.org/doi/abs/10.1145/3640457.3687092] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/]

In episode 27 of Recsperts, we meet Alessandro Piscopo, Lead Data Scientist in Personalization and Search, and Duncan Walker, Principal Data Scientist in the iPlayer Recommendations Team, both from the BBC. We discuss how the BBC personalizes recommendations across different offerings like news or video and audio content recommendations. We learn about the core values for the oldest public service media organization and the collaboration with editors in that process. The BBC once started with short video recommendations for BBC+ and nowadays has to consider recommendations across multiple domains: news, the iPlayer, BBC Sounds, BBC Bytesize, and more. With a reach of about 500M+ users who access services every week there is a huge potential. My guests discuss the challenges of aligning recommendations with public service values and the role of editors and constant exchange, alignment, and learning between the algorithmic and editorial lines of recommender systems. We also discuss the potential of cross-domain recommendations to leverage the content across different products as well as the organizational setup of teams working on recommender systems at the BBC. We learn about skews in the data due to the nature of an online service that also has a linear offering with TV and radio services. Towards the end, we also touch a bit on QUARE @ RecSys, which is the Workshop on Measuring the Quality of Explanations in Recommender Systems. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts. Don't forget to follow the podcast and please leave a review * (00:00) - Introduction * (03:10) - About Alessandro Piscopo and Duncan Walker * (14:53) - RecSys Applications at the BBC * (20:22) - Journey of Building Public Service Recommendations * (28:02) - Role and Implementation of Public Service Values * (36:52) - Algorithmic and Editorial Recommendation * (01:01:54) - Further RecSys Challenges at the BBC * (01:15:53) - Quare Workshop * (01:23:27) - Closing Remarks Links from the Episode: * Alessandro Piscopo on LinkedIn [https://www.linkedin.com/in/alessandro-piscopo-ds/] * Duncan Walker on LinkedIn [https://www.linkedin.com/in/duncan-walker/] * BBC [https://www.bbc.com/] * QUARE @ RecSys 2023 (2nd Workshop on Measuring the Quality of Explanations in Recommender Systems) [https://sites.google.com/view/quare-2023/] Papers: * Clarke et al. (2023): Personalised Recommendations for the BBC iPlayer: Initial approach and current challenges [https://dl.acm.org/doi/10.1145/3604915.3608867] * Boididou et al. (2021): Building Public Service Recommenders: Logbook of a Journey [https://dl.acm.org/doi/10.1145/3460231.3474614] * Piscopo et al. (2019): Data-Driven Recommendations in a Public Service Organisation [https://dl.acm.org/doi/abs/10.1145/3345002.3349286] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/]

In episode 26 of Recsperts, I speak with Sanne Vrijenhoek, a PhD candidate at the University of Amsterdam’s Institute for Information Law and the AI, Media & Democracy Lab. Sanne’s research explores diversity in recommender systems, particularly in the news domain, and its connection to democratic values and goals. We dive into four of her papers, which focus on how diversity is conceptualized in news recommender systems. Sanne introduces us to five rank-aware divergence metrics for measuring normative diversity and explains why diversity evaluation shouldn’t be approached blindly—first, we need to clarify the underlying values. She also presents a normative framework for these metrics, linking them to different democratic theory perspectives. Beyond evaluation, we discuss how to optimize diversity in recommender systems and reflect on missed opportunities—such as the RecSys Challenge 2024, which could have gone beyond accuracy-chasing. Sanne also shares her recommendations for improving the challenge by incorporating objectives such as diversity. During our conversation, Sanne shares insights on effectively communicating recommender systems research to non-technical audiences. To wrap up, we explore ideas for fostering a more diverse RecSys research community, integrating perspectives from multiple disciplines. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts. Don't forget to follow the podcast and please leave a review * (00:00) - Introduction * (03:24) - About Sanne Vrijenhoek * (14:49) - What Does Diversity in RecSys Mean? * (26:32) - Assessing Diversity in News Recommendations * (34:54) - Rank-Aware Divergence Metrics to Measure Normative Diversity * (01:01:37) - RecSys Challenge 2024 - Recommendations for the Recommenders * (01:11:23) - RecSys Workshops - NORMalize and AltRecSys * (01:15:39) - On the Different Conceptualizations of Diversity in RecSys * (01:28:38) - Closing Remarks Links from the Episode: * Sanne Vrijenhoek on LinkedIn [https://www.linkedin.com/in/sannevrijenhoek] * Informfully [https://www.informfully.ch/] * MIND: MIcrosoft News Dataset [https://msnews.github.io/] * RecSys Challenge 2024 [https://www.recsyschallenge.com/2024/] * NORMalize 2023: The First Workshop on the Normative Design and Evaluation of Recommender Systems [https://sites.google.com/view/normalizeworkshop/recsys23] * NORMalize 2024: The Second Workshop on the Normative Design and Evaluation of Recommender Systems [https://sites.google.com/view/normalizeworkshop] * AltRecSys 2024: The AltRecSys Workshop on Alternative, Unexpected, and Critical Ideas in Recommendation [https://altrecsys.github.io/] Papers: * Vrijenhoek et al. (2021): Recommenders with a Mission: Assessing Diversity in News Recommendations [https://dl.acm.org/doi/abs/10.1145/3406522.3446019] * Vrijenhoek et al. (2022): RADio – Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations [https://dl.acm.org/doi/abs/10.1145/3523227.3546780] * Heitz et al. (2024): Recommendations for the Recommenders: Reflections on Prioritizing Diversity in the RecSys Challenge [https://dl.acm.org/doi/10.1145/3687151.3687155] * Vrijenhoek et al. (2024): Diversity of What? On the Different Conceptualizations of Diversity in Recommender Systems [https://dl.acm.org/doi/abs/10.1145/3630106.3658926] * Helberger (2019): On the Democratic Role of News Recommenders [https://pure.uva.nl/ws/files/42634237/On_the_Democratic_Role_of_News_Recommenders.pdf] * Steck (2018): Calibrated Recommendations [https://dl.acm.org/doi/10.1145/3240323.3240372] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/]

In episode 25, we talk about the upcoming ACM Conference on Recommender Systems 2024 (RecSys) and welcome a former guest to geek about the conference. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts. Don't forget to follow the podcast and please leave a review * (00:00) - Introduction * (01:56) - Overview RecSys 2024 * (07:01) - Contribution Stats * (09:37) - Interview Links from the Episode: * RecSys 2024 Conference Website [https://recsys.acm.org/recsys24/] Papers: * RecSys '24: Proceedings of the 18th ACM Conference on Recommender Systems [https://dl.acm.org/doi/proceedings/10.1145/3640457] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/]