The Podcast - Unlocking Engineering Excellence: Navigating Success with GitHub's ESSP in the Age of AI
Are you an engineering leader trying to understand how to measure the true impact of your teams and tools, especially in the era of AI? Balancing developer happiness, velocity, and quality isn't easy, and success isn't driven by just one of these factors in isolation. It's when these three foundational zones work in unison that organizations achieve their best results and see improved business outcomes.
In this episode, we dive into GitHub’s Engineering System Success Playbook (ESSP), a structured yet flexible framework designed to help organizations drive meaningful, measurable improvements and align engineering practices with business goals. Inspired by leading frameworks like SPACE, DORA, and DevEx, the ESSP offers a balanced and comprehensive approach to understanding your engineering system.
The ESSP centers on a systems thinking approach, recognizing that engineering is an interconnected ecosystem. It prioritizes long-term, sustainable improvements over quick wins that might cause negative downstream effects.
The playbook outlines a three-step process to achieve engineering success:
* Step 1: Identify the current barriers to success. This involves understanding your current state and desired future state, auditing processes, gathering data (both quantitative and qualitative), understanding organizational priorities, and conducting research to find friction points. Prioritizing the most impactful barriers and target metrics is key.
* Step 2: Evaluate what needs to be done to achieve your goals. Here, you identify potential solutions, estimate their cost and resource requirements, and conduct a risk, cost, and benefit analysis. Engaging stakeholders, potentially starting with pilots, and creating mitigation plans are crucial parts of this step.
* Step 3: Implement your changes, monitor results, and adjust. This involves assigning ownership, fostering communication and transparency, training teams, and monitoring performance using both metrics and qualitative feedback. Analyzing results, pivoting if needed, and maintaining continuous feedback loops are essential for sustained progress.
At the heart of the ESSP are 12 key metrics (three for each zone: Quality, Velocity, Developer Happiness, and Business Outcomes) that you can track to measure performance over time. These are primarily downstream or lagging metrics, but the playbook emphasizes complementing them with leading indicators that provide early signals of friction or potential impact.
Examples of the recommended metrics include:
* Quality: Change failure rate, Failed deployment recovery time, Code security and maintainability.
* Velocity: Lead time, Deployment frequency, PRs merged per developer.
* Developer Happiness: Flow state experience, Engineering tooling satisfaction, Copilot satisfaction.
* Business Outcomes: AI leverage, Engineering expenses to revenue, Feature engineering expenses to total engineering expenses.
The ESSP highlights the importance of using these metrics carefully, focusing on improvements over time rather than benchmarks, and avoiding the pitfalls of gamification that can incentivize undesirable behaviors. Instead, thoughtful gamification can align with intrinsic motivators like mastering skills or contributing to team goals.
Beyond the core steps, the playbook stresses concepts like tailoring the approach to your team's needs, embracing change management frameworks, fostering a growth mindset that values learning from every attempt, and most importantly, involving your engineering teams early in the process. Engineering is a team sport, and metrics should focus on teams and organizations, not individual developers.
Ultimately, the ESSP provides a way to connect engineering improvements to business-level outcomes. Faster delivery and lower change failure rates link to market responsiveness and customer trust. Happier developers contribute to productivity and innovation. Metrics like AI leverage help quantify the ROI of investments like GitHub Copilot. This holistic view helps engineering leaders demonstrate tangible value to the business.
Whether you're adopting AI tools or tackling existing bottlenecks, the ESSP offers a blueprint to systematically increase developer joy, ship value faster, and raise the quality bar. Learn how to use this playbook to unlock your engineering team's full potential.Get the deep dive blog post here [https://galacticnaut.substack.com/p/engineering-success-playbook-ai-era].
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit galacticnaut.substack.com [https://galacticnaut.substack.com?utm_medium=podcast&utm_campaign=CTA_1]
