Recsperts - Recommender Systems Experts

#32: RecSys in the Delivery Industry at Wolt with Sasha Fedintsev

In episode 32 of Recsperts, I’m joined by my colleague Sasha Fedintsev, Staff Applied Scientist at Wolt (DoorDash), working across personalization and ads, to unpack the realities of building large-scale recommender systems in food, grocery, and retail delivery. Together, we discuss the specifics of personalization in the delivery domain, and the models and ideas that power Wolt’s recommender system across 30+ markets - where theory quickly meets messy, high-stakes practice. We explore what makes this domain fundamentally different from traditional e-commerce: strong locality constraints, real-time context, and a heavy skew toward repurchasing behavior. Sasha explains how these factors break many textbook approaches - like standard collaborative filtering - and require creative adaptations such as clustering strategies and multi-stage ranking systems optimized for latency, all while respecting locality constraints. We also discuss the evolution of recommendation approaches over time - from classical collaborative filtering with ALS, to Neural Collaborative Filtering with BPR, and ultimately to transformer-based models for user sequence modeling and next-purchase prediction powering today’s venue ranking systems. We also touch on practical challenges such as evaluation in real-world systems, including A/B testing pitfalls and biases in logged data, as well as the complexity introduced by multi-surface experiences like discovery pages, vertical lists, and search. Beyond venues, we discuss why item-level recommendation is an order of magnitude harder - due to scale, context dependence, and availability constraints - and what this implies for future system design. Throughout the episode, Sasha provides a candid view on the evolving role of a Staff Applied Scientist - bridging research and production, setting scientific standards, and driving cross-team impact. Enjoy this enriching episode of RECSPERTS – Recommender Systems Experts. Don’t forget to follow the podcast and please leave a review. * (00:00) - Introduction * (05:10) - About Sasha Fedintsev * (15:26) - The Role of a Staff Applied Scientist * (25:50) - Challenges and Specifics of the Delivery Industry * (47:24) - Ranking and Recommendation Problems at Wolt * (51:31) - NCF with BPR for Wolt's First DNN Recommendation Model * (01:16:43) - User Sequence Transformers for Next Purchase Prediction * (01:26:51) - Explore vs. Exploit or New vs. Recurring Purchases * (01:31:29) - Ads Personalization at Wolt * (01:36:16) - Further Challenges in RecSys * (01:37:58) - A Final Note on Radical Longevity * (01:46:30) - Closing Remarks Links from the Episode: * Alexander "Sasha" Fedintsev on LinkedIn [https://www.linkedin.com/in/fedintsevalex/] * Alexander on X [https://x.com/afedintsev] * Wolt [https://wolt.com/en/discovery] * Alexander Fedintsev at Wolt Tech Talks: Restaurant discovery with Wolt: Deep Neural Networks to power recommendations [https://www.youtube.com/watch?v=MHScrLkIDq0] * H3 Geospatial Indexing System [https://h3geo.org/] * Recommenders Repository [https://github.com/recommenders-team/recommenders] * Tanja Reilly: The Staff Engineer's Path [https://www.oreilly.com/library/view/the-staff-engineers/9781098118723/] * Will Larson: Staff Engineer: Leadership beyond the management track [https://staffeng.com/book/] * Coupon collector's problem [https://en.wikipedia.org/wiki/Coupon_collector%27s_problem] * Alexander Fedintsev (2026): Longevity Bottlenecks: Part I — Dementia [https://rlegroup.net/2026/01/04/longevity-bottlenecks-part-i-dementia/] Papers: * Rendle et al. (2009): BPR: Bayesian personalized ranking from implicit feedback [https://dl.acm.org/doi/10.5555/1795114.1795167] * He et al. (2017): Neural Collaborative Filtering [https://dl.acm.org/doi/10.1145/3038912.3052569] * Dacrema et al. (2019): Are we really making much progress? A worrying analysis of recent neural recommendation approaches [https://dl.acm.org/doi/10.1145/3298689.3347058] * Rendle et al (2020): Neural Collaborative Filtering vs. Matrix Factorization Revisited [https://dl.acm.org/doi/abs/10.1145/3383313.3412488] * Hu et al. (2008): Collaborative Filtering for Implicit Feedback Datasets [https://dl.acm.org/doi/10.1109/ICDM.2008.22] * Grbovic et al. (2015): E-commerce in Your Inbox: Product Recommendations at Scale [https://dl.acm.org/doi/10.1145/2783258.2788627] * Quadrana et al. (2018): Sequence-Aware Recommender Systems [https://dl.acm.org/doi/10.1145/3190616] * Su et al. (2024): Long-Term Value of Exploration: Measurements, Findings and Algorithms [https://dl.acm.org/doi/abs/10.1145/3616855.3635833] * Tran et al. (2024): Transformers Meet ACT-R: Repeat-Aware and Sequential Listening Session Recommendation [https://dl.acm.org/doi/abs/10.1145/3640457.3688139] * Lichtenberg et al. (2024): Ranking Across Different Content Types: The Robust Beauty of Multinomial Blending [https://dl.acm.org/doi/abs/10.1145/3640457.3688059] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/]

#31: Psychology-Aware Recommender Systems with Elisabeth Lex

In episode 31 of Recsperts, I sit down with Elisabeth Lex, Full Professor of Human-Computer Interfaces and Inclusive Technologies at Graz University of Technology and a leading researcher at the intersection of recommender systems, psychology, and human–computer interaction. Together, we explore how recommender systems can become truly human-centric by integrating cognitive, emotional, and personality-aware models into their design. Elisabeth begins by addressing a common reductionism in the field: treating users primarily as data points rather than as humans with goals, emotions, memories, and cognitive boundaries. We revisit the origins of psychology-informed recommendation, including the Grundy system -the first recommender system, built nearly 50 years ago - which framed book recommendation through stereotype modeling. From there, we discuss how the community’s focus shifted toward solving recommendation mainly as an algorithmic optimization problem, often sidelining richer models of human decision-making. We then map out the three major branches of psychology-informed RecSys - cognition-inspired, affect-aware, and personality-aware - and dive into practical examples. Elisabeth walks us through her work on modeling music re-listening behavior using cognitive architectures such as ACT-R (Adaptive Control of Thought–Rational) and shows how cognitive constructs like memory decay, attention, and familiarity can meaningfully augment standard approaches like collaborative filtering. We also explore how hybrid systems that combine cognitive models with collaborative filtering can yield not just higher accuracy but also more novelty, diversity, and clearer explanations. Our conversation also turns to user-centric evaluation. Elisabeth argues that accuracy metrics alone cannot tell us whether a system is genuinely helpful. Instead, we must measure attitudes, perceptions, motivations, and emotional responses - while carefully accounting for cognitive biases, UI effects, and users’ lived experiences. Towards the end, Elisabeth discusses emerging research directions such as hybrid AI (symbolic + sub-symbolic methods), the role of LLMs and agents, the risks of replacing human studies with automated evaluations, and the responsibility our community has to understand users beyond their clicks. Enjoy this enriching episode of RECSPERTS – Recommender Systems Experts. Don’t forget to follow the podcast and please leave a review. * (00:00) - Introduction * (03:15) - About Elisabeth Lex * (07:55) - Grundy, the first Recommender System * (09:03) - Bridging the Gap between Psychology and Modern RecSys * (17:21) - On how and when Elisabeth became a Researcher * (21:39) - Survey on Psychology-Informed RecSys * (39:29) - Personality-Aware Recommendation * (49:43) - Affect- and Emotion-Aware Recommendation * (01:01:37) - Cognition-Inspired Recommendation and the ACT-R Framework * (01:14:39) - Combining Collaborative Filtering and ACT-R for Explainability * (01:21:26) - Human-Centered Design * (01:26:15) - Further Challenges and Closing Remarks Links from the Episode: * Elisabeth Lex on LinkedIn [https://www.linkedin.com/in/elisabeth-lex-6457a427b/] * Website of Elisabeth [http://www.elisabethlex.info/] * AI for Society Lab [https://aisocietylab.github.io] * First International Workshop on Recommender Systems for Sustainability and Social Good | co-located with RecSys 2024 [https://recsogood.github.io/recsogood24/index.html] * Second International Workshop on Recommender Systems for Sustainability and Social Good | co-located with RecSys 2025 [https://recsogood.github.io/recsogood25/] * HyPer Workshop: Hybrid AI for Human-Centric Personalization [https://aisocietylab.github.io/hyper/] * Tutorial on Psychology-Informed RecSys [https://www.youtube.com/watch?v=ZhGAZ8w_6jI] * ACT-R: Adaptive Control of Thought-Rational [https://de.wikipedia.org/wiki/ACT-R] * POPROX: Platform for OPen Recommendation and Online eXperimentation [https://poprox.ai/] Papers: * Elaine Rich (1979): User Modeling via Stereotypes [https://www.cs.utexas.edu/~ear/CogSci.pdf] * Lex et al. (2021): Psychology-informed Recommender Systems [https://elisabethlex.info/docs/2021fntir-psychology.pdf] * Reiter-Haas et al. (2021): Predicting Music Relistening Behavior Using the ACT-R Framework [https://dl.acm.org/doi/10.1145/3460231.3478846] * Moscati et al. (2023): Integrating the ACT-R Framework with Collaborative Filtering for Explainable Sequential Music Recommendation [https://dl.acm.org/doi/10.1145/3604915.3608838] * Tran et al. (2024): Transformers Meet ACT-R: Repeat-Aware and Sequential Listening Session Recommendation [https://dl.acm.org/doi/abs/10.1145/3640457.3688139] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/]

#30: Serendipity for Recommender Systems with Annelien Smets

In episode 30 of Recsperts, I speak with Annelien Smets, Professor at Vrije Universiteit Brussel and Senior Researcher at imec-SMIT, about the value, perception, and practical design of serendipity in recommender systems. Annelien introduces her framework for understanding serendipity through intention, experience, and affordances, and explains the paradox of artificial serendipity - why it cannot be engineered, but only designed for. We start by unpacking the paradox of serendipity: while serendipity cannot be engineered or planned, systems and environments can be designed to increase the likelihood that serendipitous experiences occur. Annelien explains why randomness alone is not enough and why serendipity always emerges from an interplay between an unexpected encounter and a user’s ability to recognize its relevance and value. A central part of our discussion focuses on Annelien’s recent framework that distinguishes between intended, experienced, and afforded serendipity. We explore why organizations first need to clarify why they want serendipity - whether as an ideal, a common good, a mediator to achieve other goals (such as long-term retention or long-tail exposure), or even as a product feature in itself. From there, we dive into how users actually experience serendipity, drawing on qualitative interview research that identifies three core components: encounters must feel fortuitous, refreshing, and enriching. These components can manifest in different “flavors,” such as taste broadening, taste deepening, or rediscovering forgotten interests. We then move beyond algorithms to discuss affordances for serendipity - design principles that span content, user interfaces, and information access. Using examples from libraries, urban spaces, and digital platforms, Annelien shows why serendipity is a system-level property rather than a single metric or model tweak. We also discuss where serendipity can go wrong, including the Netflix “Surprise Me” feature, and why mismatched expectations can actually harm user experience. To close, we reflect on open research questions, from measuring different types of serendipity to understanding how content types, business models, and platform economics shape what is possible. Annelien also challenges a common myth: serendipity does not automatically burst filter bubbles—and should not be treated as a silver bullet. Enjoy this enriching episode of RECSPERTS – Recommender Systems Experts. Don’t forget to follow the podcast and please leave a review. * (00:00) - Introduction * (03:57) - About Annelien Smets * (14:42) - Paradox and Definition of (Artificial) Serendipity * (27:04) - Intended Serendipity * (43:01) - Experienced Serendipity * (01:01:18) - Afforded Serendipity * (01:13:49) - Examples of Serendipity Going Wrong * (01:17:40) - Framework for Serendipity * (01:22:41) - Further Challenges and Closing Remarks Links from the Episode: * Annelien Smets on LinkedIn [https://www.linkedin.com/in/anneliensmets/] * Website of Annelien [https://www.anneliensmets.be/] * LinkedIn Article by Annelien Smets (2025): Overcoming the Paradox of Artificial Serendipity [https://www.linkedin.com/pulse/overcoming-paradox-artificial-serendipity-annelien-smets-qn2ue/?trackingId=LePKiYfUh4N5CHhBKA9zHQ%3D%3D] * The Serendipity Society [https://www.theserendipitysociety.org/] * Serendipity Engine [https://www.serendipityengine.be/] Papers: * Smets (2025): Intended, afforded, and experienced serendipity: overcoming the paradox of artificial serendipity [https://link.springer.com/article/10.1007/s10676-025-09841-6] * Smets et al. (2022): Serendipity in Recommender Systems Beyond the Algorithm: A Feature Repository and Experimental Design [https://ceur-ws.org/Vol-3222/paper4.pdf] * Binst et al. (2025): What Is Serendipity? An Interview Study to Conceptualize Experienced Serendipity in Recommender Systems [https://dl.acm.org/doi/10.1145/3699682.3728325] * Ziarani et al. (2021): Serendipity in Recommender Systems: A Systematic Literature Review [https://link.springer.com/article/10.1007/s11390-020-0135-9] * Chen et al. (2021): Values of User Exploration in Recommender Systems [https://dl.acm.org/doi/10.1145/3460231.3474236] * Smets et al. (2025): Why Do Recommenders Recommend? Three Waves of Research Perspectives on Recommender Systems [https://ceur-ws.org/Vol-4063/paper3.pdf] * Smets (2023): Designing for Serendipity, a Means or an End? [https://cris.vub.be/ws/portalfiles/portal/87378097/Smets_2022_JDOC_AAM_online.pdf] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/]

#29: Transformers for Recommender Systems with Craig Macdonald and Sasha Petrov

In episode 29 of Recsperts, I welcome Craig Macdonald, Professor of Information Retrieval at the University of Glasgow, and Aleksandr “Sasha” Petrov, PhD researcher and former applied scientist at Amazon. Together, we dive deep into sequential recommender systems and the growing role of transformer models such as SASRec and BERT4Rec. Our conversation begins with their influential replicability study of BERT4Rec, which revealed inconsistencies in reported results and highlighted the importance of training objectives over architecture tweaks. From there, Craig and Sasha guide us through their award-winning research on making transformers for sequential recommendation with large corpora both more effective and more efficient. We discuss how recency sampling (RSS) reduces training times dramatically, and how gSASRec overcomes the problem of overconfidence in models trained with negative sampling. By generalizing the sigmoid function (gBCE), they were able to reconcile cross-entropy–based optimization results with negative sampling, matching the effectiveness of softmax approaches while keeping training scalable for large corpora. We also explore RecJPQ, their recent work on joint product quantization for item embeddings. This approach makes transformer-based sequential recommenders substantially faster at inference and far more memory-efficient for embeddings—while sometimes even improving effectiveness thanks to regularization effects. Towards the end, Craig and Sasha share their perspective on generative approaches like GPTRec, the promises and limits of large language models in recommendation, and what challenges remain for the future of sequential recommender systems. Enjoy this enriching episode of RECSPERTS – Recommender Systems Experts. Don’t forget to follow the podcast and please leave a review. * (00:00) - Introduction * (04:09) - About Craig Macdonald * (04:46) - About Sasha Petrov * (13:48) - Tutorial on Transformers for Sequential Recommendations * (19:24) - SASRec vs. BERT4Rec * (21:25) - Replicability Study of BERT4Rec for Sequential Recommendation * (32:52) - Training Sequential RecSys using Recency Sampling * (40:01) - gSASRec for Reducing Overconfidence by Negative Sampling * (01:00:51) - RecJPQ: Training Large-Catalogue Sequential Recommenders * (01:21:37) - Generative Sequential Recommendation with GPTRec * (01:29:12) - Further Challenges and Closing Remarks Links from the Episode: * Craig Macdonald on LinkedIn [https://www.linkedin.com/in/craigmacdonald/] * Sasha Petrov on LinkedIn [https://www.linkedin.com/in/asash/] * Sasha's Website [https://asash.github.io/] * Tutorial: Transformers for Sequential Recommendation (ECIR 2024) [https://github.com/asash/transformers-for-recsys-tutorial] * Tutorial Recording from ACM European Summer School in Bari (2024) [https://www.youtube.com/watch?v=H732E7qzbis&ab_channel=ACMEuropeanSummerSchoolinRecSys] * Talk: Neural Recommender Systems (European Summer School in Information Retrieval 2024) [https://www.youtube.com/watch?v=Tan2o8osnW8&ab_channel=MohammadA] Papers: * Kang et al. (2018): Self-Attentive Sequential Recommendation [https://arxiv.org/abs/1808.09781] * Sun et al. (2019): BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer [https://arxiv.org/pdf/1904.06690] * Petrov et al. (2022): A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation [https://eprints.gla.ac.uk/275645/2/275645.pdf] * Petrov et al. (2022): Effective and Efficient Training for Sequential Recommendation using Recency Sampling [https://dl.acm.org/doi/10.1145/3523227.3546785] * Petrov et al. (2024): RSS: Effective and Efficient Training for Sequential Recommendation Using Recency Sampling (extended version) [https://dl.acm.org/doi/10.1145/3604436] * Petrov et al. (2023): gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling [https://eprints.gla.ac.uk/301348/3/301348.pdf] * Petrov et al. (2025): Improving Effectiveness by Reducing Overconfidence in Large Catalogue Sequential Recommendation with gBCE loss [https://dl.acm.org/doi/full/10.1145/3699521] * Petrov et al. (2024): RecJPQ: Training Large-Catalogue Sequential Recommenders [https://dl.acm.org/doi/10.1145/3616855.3635821] * Petrov et al. (2024): Efficient Inference of Sub-Item Id-based Sequential Recommendation Models with Millions of Items [https://dl.acm.org/doi/10.1145/3640457.3688168] * Rajput et al. (2023): Recommender Systems with Generative Retrieval [https://papers.neurips.cc/paper_files/paper/2023/file/20dcab0f14046a5c6b02b61da9f13229-Paper-Conference.pdf] * Petrov et al. (2023): Generative Sequential Recommendation with GPTRec [https://arxiv.org/abs/2306.11114] * Petrov et al. (2024): Aligning GPTRec with Beyond-Accuracy Goals with Reinforcement Learning [https://eprints.gla.ac.uk/322171/1/322171.pdf] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/] Disclaimer: Craig holds concurrent appointments as a Professor of Information Retrieval at University of Glasgow and as an Amazon Scholar. This podcast describes work performed at the University of Glasgow and is not associated with Amazon.

#28: Multistakeholder Recommender Systems with Robin Burke

In episode 28 of Recsperts, I sit down with Robin Burke, professor of information science at the University of Colorado Boulder and a leading expert with over 30 years of experience in recommender systems. Together, we explore multistakeholder recommender systems, fairness, transparency, and the role of recommender systems in the age of evolving generative AI. We begin by tracing the origins of recommender systems, traditionally built around user-centric models. However, Robin challenges this perspective, arguing that all recommender systems are inherently multistakeholder—serving not just consumers as the recipients of recommendations, but also content providers, platform operators, and other key players with partially competing interests. He explains why the common “Recommended for You” label is, at best, an oversimplification and how greater transparency is needed to show how stakeholder interests are balanced. Our conversation also delves into practical approaches for handling multiple objectives, including reranking strategies versus integrated optimization. While embedding multistakeholder concerns directly into models may be ideal, reranking offers a more flexible and efficient alternative, reducing the need for frequent retraining. Towards the end of our discussion, we explore post-userism and the impact of generative AI on recommendation systems. With AI-generated content on the rise, Robin raises a critical concern: if recommendation systems remain overly user-centric, generative content could marginalize human creators, diminishing their revenue streams.   Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts. Don't forget to follow the podcast and please leave a review * (00:00) - Introduction * (03:24) - About Robin Burke and First Recommender Systems * (26:07) - From Fairness and Advertising to Multistakeholder RecSys * (34:10) - Multistakeholder RecSys Terminology * (40:16) - Multistakeholder vs. Multiobjective * (42:43) - Reciprocal and Value-Aware RecSys * (59:14) - Objective Integration vs. Reranking * (01:06:31) - Social Choice for Recommendations under Fairness * (01:17:40) - Post-Userist Recommender Systems * (01:26:34) - Further Challenges and Closing Remarks Links from the Episode: * Robin Burke on LinkedIn [https://www.linkedin.com/in/robin-burke-b878303] * Robin's Website [https://www.that-recsys-lab.net/home/people/burke] * That Recommender Systems Lab [https://www.that-recsys-lab.net/] * Reference to Broder's Keynote on Computational Advertising and Recommender Systems from RecSys 2008 [https://recsys.acm.org/recsys08/keynotes/] * Multistakeholder Recommender Systems (from Recommender Systems Handbook), chapter by Himan Abdollahpouri & Robin Burke [https://link.springer.com/chapter/10.1007/978-1-0716-2197-4_17] * POPROX: The Platform for OPen Recommendation and Online eXperimentation [https://poprox.ai/] * AltRecSys 2024 (Workshop at RecSys 2024) [https://altrecsys.github.io/] Papers: * Burke et al. (1996): Knowledge-Based Navigation of Complex Information Spaces [https://cdn.aaai.org/AAAI/1996/AAAI96-069.pdf] * Burke (2002): Hybrid Recommender Systems: Survey and Experiments [https://link.springer.com/article/10.1023/A:1021240730564] * Resnick et al. (1997): Recommender Systems [https://dl.acm.org/doi/10.1145/245108.245121] * Goldberg et al. (1992): Using collaborative filtering to weave an information tapestry [https://dl.acm.org/doi/10.1145/138859.138867] * Linden et al. (2003): Amazon.com Recommendations - Item-to-Item Collaborative Filtering [https://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf] * Aird et al. (2024): Social Choice for Heterogeneous Fairness in Recommendation [https://dl.acm.org/doi/10.1145/3640457.3691706] * Aird et al. (2024): Dynamic Fairness-aware Recommendation Through Multi-agent Social Choice [https://dl.acm.org/doi/full/10.1145/3690653] * Burke et al. (2024): Post-Userist Recommender Systems : A Manifesto [https://arxiv.org/abs/2410.11870] * Baumer et al. (2017): Post-userism [https://dl.acm.org/doi/10.1145/3025453.3025740] * Burke et al. (2024): Conducting Recommender Systems User Studies Using POPROX [https://dl.acm.org/doi/abs/10.1145/3640457.3687092] General Links: * Follow me on LinkedIn [https://www.linkedin.com/in/marcel-kurovski/] * Follow me on X [https://twitter.com/MarcelKurovski] * Send me your comments, questions and suggestions to marcel.kurovski@gmail.com [marcel.kurovski@gmail.com] * Recsperts Website [https://www.recsperts.com/]