Data Processing Evolved: OpenLineage with Willy Lulciuc

Data Processing Evolved: OpenLineage with Willy Lulciuc

Willy Lulciuc (@wslulciuc [https://x.com/wslulciuc?lang=en]) is a pioneer in data engineering and one of the creators of OpenLineage, the open-source framework for data lineage collection and analysis. It enables consistent collection of lineage metadata, giving engineers a better perspective on how data is produced and used, so they can better solve complex problems. Join us to learn about the evolution of data processing, the role of AI, and the future of data engineering. Subscribe to Contributor on Substack [https://contributor.substack.com/] for email notifications! In this episode we discuss: * Willy’s journey from WeWork to oleander * The role of AI in data engineering * Challenges in data processing and observability * A vision for the future of data platforms Links: * OpenLineage [https://openlineage.io/] * oleander [https://oleander.dev/] People mentioned: * Julien Le Dem (@J_ [https://x.com/J_])

Revolutionizing Computer Vision: OpenFilter with Andrew Smith

Andrew Smith (Github: @asmith-plainsight [https://github.com/asmith-plainsight]) dives deep into OpenFilter, the open-source framework for building computer vision workflows. Andrew is the CTO of Plainsight which is a leader in modern computer vision infrastructure. Tune in to find out how OpenFilter is simplifying and revolutionizing computer vision applications. Subscribe to Contributor on Substack [https://contributor.substack.com/] for email notifications! In this episode we discuss: * The challenges in deploying computer vision, particularly in edge environments * How Andrew’s time at Google influenced the development of OpenFilter * Three problems Andrew sought to solve before open-sourcing the project * Best use cases for OpenFilter * The surprising nascence of computer vision applications Links: * OpenFilter [https://openfilter.io/] * Plainsight [https://plainsight.ai/] * OpenLineage [https://openlineage.io/] People mentioned: * Kit Merker (@KitMerker [https://x.com/kitmerker?lang=en]) * Willy Lulciuc (@wslulciuc [https://x.com/wslulciuc?lang=en])

Big Complexity Revisited: Dagster with Pete Hunt

Pete Hunt (@floydophone [https://x.com/floydophone?lang=en]) joins Eric Anderson (@ericmander [https://x.com/ericmander]) to talk about Dagster, the open-source data orchestration platform built for productivity. In 2021, Eric interviewed Dagster’s founder, Nick Schrock, about an earlier iteration of the product. Now, four years later, Pete has become the CEO of Dagster Labs and has all the updates on how this platform has changed along with the industry and the world. Subscribe to Contributor on Substack [https://contributor.substack.com/] for email notifications! In this episode we discuss: * How and why Dagster has invested in software-defined data assets * Nailing a layered approach between full-code engineering and low-code solutions * The changing role of AI in data pipelines * Dagster’s renewed effort to build observability tools Links: * Dagster [https://dagster.io/] People mentioned: * Nick Schrock (@schrockn [https://x.com/schrockn]) Other episodes: * Dagster with Nick Schrock [https://www.contributor.fyi/dagster]

Simplifying Distributed Systems: Dapr with Mark Fussell

Mark Fussell (@mfussell [https://x.com/mfussell]) is the co-creator of Dapr, the open-source runtime system designed to support cloud native and serverless computing. Dapr provides APIs that simplify the development of distributed applications, providing essential functionalities like service invocation, pub/sub messaging, and observability.  Subscribe to Contributor on Substack [https://contributor.substack.com/] for email notifications! In this episode we discuss: * The difference between Dapr and a service mesh * How Dapr is beneficial for platform engineering teams * From Microsoft to Diagrid in the startup world * The significance of Dapr’s graduation from the CNCF Links: * Dapr [https://dapr.io/] * Diagrid [https://www.diagrid.io/] * KEDA [https://keda.sh/] People mentioned: * Yaron Schneider (@yaronschneider [https://x.com/yaronschneider/]) Other episodes: * Istio with Sven Mawson [https://www.contributor.fyi/istio] * LinkerD with William Morgan [https://www.contributor.fyi/linkerd] * Robust Observability: OpenTelemetry with Austin Parker [https://www.contributor.fyi/opentelemetry]

Messages, Not Metadata: Session with Kee Jefferys

Kee Jefferys (@JefferysKee [https://x.com/jefferyskee]) is the technical co-founder of Session, the end-to-end encrypted messenger that emphasizes user confidentiality and anonymity. Session uses a blockchain-based decentralized network for message transmission and is open-source, so the system can be run entirely by its community. In this episode, Kee explains the importance of reducing metadata, the challenges of centralization, and the role of incentives in creating a decentralized network.  Contributor is looking for a community manager! If you want to know more, shoot us an email at eric@scalevp.com [eric@scalevp.com]. Subscribe to Contributor on Substack [https://contributor.substack.com/] for email notifications! In this episode we discuss: * Why the demand for better privacy has increased  * How Session differs from other encrypted messaging platforms  * Combining encryption and incentive models * Why open-source is so critical to messaging applications trying to prove their security * How Session could outlast its developers Links: * Session [https://getsession.org/]