Core Elements of Platform Engineering.

This post will help to demystify the core concepts of Platform Engineering. We have broken down the essentials in a way that's easy to grasp, even if you’re just getting started.

We'll dive into what it means to treat your platform as a product, why having an Internal Developer Platform (IDP) makes life easier, how the right team structure boosts efficiency, and what “Golden Paths” are all about.

By the end, you'll have a solid understanding of these foundational ideas and, hopefully, some inspiration to start thinking about a platform strategy for your own organization.

Introducing the core elements of Platform Engineering

Platform Engineering introduces several core elements designed to create a more efficient, developer-friendly environment:

1. Product mindset

• Treating the platform as a product means continuously iterating based on user feedback and performance metrics.
• This ensures the platform remains relevant, user-friendly, and capable of meeting organizational needs over time.

2. Internal Developer Platform (IDP)

• The IDP is the centerpiece of Platform Engineering, serving as a centralized hub where developers can access all necessary resources, from infrastructure to observability tools. By streamlining access, the IDP enhances productivity by reducing the time spent on non-core tasks.

3. A Platform team and a new teams topology

• Effective team collaboration is key to Platform Engineering. By structuring teams according to their functions and workflows, Platform Engineering reduces cognitive load and empowers stream-aligned teams to focus on delivering business value by abstracting away complexity.
• A unified platform team enables cohesive and consistent platform-related decisions and developments across the organization.

4. Golden Paths or Abstractions

• These are predefined, standardized workflows that simplify common tasks. By providing best-practice approaches.
• Golden Paths are reliable and scalable foundations to reduce the decision-making burden on developers, speeding up the development process and ensuring consistency.

Product Mindset

Adopting a product mindset for the platform means treating it as a continuously evolving service rather than a one-time project.

This approach focuses on meeting the needs of internal users—primarily development teams—through regular updates and improvements based on user feedback and performance metrics.

Key Elements of a Product Mindset

• User-Centric Design: Build the platform with the end-users in mind. Engage regularly with developers to understand their needs and pain points, ensuring the platform truly supports their workflows.
• Continuous Feedback Loop: Regularly collect and act on user feedback to inform the platform’s development roadmap. This keeps the platform aligned with actual user needs.
• Performance Metrics: Track key performance indicators (KPIs) like adoption rates, time-to-market, and developer satisfaction. Use this data to guide decisions and prioritize improvements.
• Iterative Development: Continuously release updates and new features, each informed by user input and performance data. This agile approach ensures the platform evolves to meet changing needs.

Developer Experience

Just as User Experience (UX) aims to minimize friction between a tool and its user's goals by eliminating obstacles, detours, and discomforts, Developer Experience (DevEx) focuses on enhancing the productivity and satisfaction of developers.

In the context of a platform-as-a-product, DevEx is about creating an environment where developers can work efficiently and effectively, with minimal friction between the platform and the ultimate goal of continuous value delivery.

Developer Experience (DevEx) is essential in treating the platform as a product, focusing on removing any barriers that might slow down or frustrate developers. This involves automating repetitive tasks, simplifying deployment processes, and ensuring all necessary resources are easily accessible. By streamlining workflows through intuitive interfaces, seamless integrations, and well-documented processes, the platform can minimize friction, enabling developers to achieve their goals with minimal effort and ultimately enhancing productivity and satisfaction.

Ensuring Long-Term Relevance

A platform treated as a product remains adaptable and scalable. By regularly engaging with users and aligning platform developments with business goals, the platform stays relevant and valuable over time. This approach not only enhances user satisfaction but also ensures the platform remains a critical enabler of business success.

Internal Developer Platform

Internal Developer Platforms (IDPs) are the cornerstone of Platform Engineering, acting as a centralized hub where developers can access all the resources they need, from infrastructure to observability tools. By streamlining access to these resources, IDPs significantly enhance developer productivity, allowing teams to focus on core tasks rather than getting bogged down by operational complexities.

Evan Bottcher encapsulates this concept by stating, “A digital platform is a foundation of self-service APIs, tools, services, knowledge, and support which are arranged as a compelling internal product.”

As the centerpiece of Platform Engineering, IDPs enables a self-service experience for the platform capabilities tailored to internal teams' needs. They streamline workflows, reduce operational overhead, and accelerate time-to-market, making them essential for modern software development.

Platform team and topology

The concept of the platform team and its role within an organization was thoroughly explored in the influential book Team Topologies by Matthew Skelton and Manuel Pais, published in 2019.

This book introduced the idea of a unified platform team as a cornerstone for effective software delivery in modern organizations.

Introducing the Unified Platform Team

In Team Topologies, Skelton and Pais emphasize that a unified platform team is essential for creating a strong foundation upon which product teams can build and deliver value more efficiently. The mission of the platform team is to reduce the cognitive load on product teams by providing a set of well-defined, self-service capabilities that are easy to consume. This allows product teams to focus on their core mission—delivering features and improvements to end-users—without getting bogged down by the complexities of underlying infrastructure.

Mission of the Platform Team

As organizations scale, the decentralized nature of traditional DevOps practices often leads to inefficiencies and complexity. Platform engineering offers a solution by providing a more organized, centralized approach to managing the software development lifecycle. This centralization is vital in handling the intricacies of modern development environments, where maintaining agility and competitiveness is paramount.

The platform team's mission extends beyond merely providing tools or infrastructure. It involves creating a comprehensive Internal Developer Platform (IDP) that abstracts and simplifies the complexities of software delivery. Key responsibilities of the platform team include:

• Enabling Self-Service: The platform team provides self-service capabilities that allow product teams to deploy, manage, and monitor their applications without needing to rely on other teams.
• Standardization and Consistency: By offering standardized tools, processes, and environments, the platform team ensures that all teams work within a consistent framework, reducing variability and the risk of errors.
• Reducing Cognitive Load: The platform team takes on the complexity of managing infrastructure, CI/CD pipelines, and other foundational elements, so product teams can focus on feature development and innovation.
• Driving Continuous Improvement: The platform team continuously iterates on the platform, incorporating feedback from users (product teams) to improve the platform's capabilities and user experience.

Collaboration in the New Team Topology

In the team topology proposed by Team Topologies, the platform team collaborates closely with other types of teams within the organization, primarily stream-aligned teams and enabling teams:

• Stream-Aligned Teams: These are product-focused teams that deliver features and services directly to end-users. The platform team supports these teams by providing the necessary tools and infrastructure to develop, test, and deploy their applications efficiently. The platform team’s role is to ensure that stream-aligned teams can operate with minimal friction.
• Enabling Teams: These teams specialize in areas such as DevOps, security, or UX and work alongside both platform and stream-aligned teams to ensure that best practices are followed. The platform team collaborates with enabling teams to embed their expertise into the platform, ensuring that it meets the organization’s technical and compliance requirements.

The platform team must also maintain open lines of communication with all other teams to ensure that the platform evolves in alignment with the needs of the organization. Regular feedback loops, workshops, and collaborative planning sessions are essential for keeping the platform relevant and effective.

A New Collaboration Model

Team Topologies introduces a new collaboration model that revolves around fast flow of change, responsibility alignment, and well-defined team interactions. The platform team acts as an internal service provider, but with a strong product mindset, where the platform is treated as a product with its own roadmap, user experience considerations, and continuous improvement cycle.

In this model, the platform team is not a gatekeeper but an enabler. The goal is to empower other teams to deliver value autonomously while ensuring that the organization’s standards and practices are consistently applied across all projects.

Golden Paths

A crucial component of Platform Engineering is the concept of Golden Paths. Golden Paths are a method for exposing platform capabilities and procedures in a way that makes them easily consumable by internal users. The goal is to simplify and streamline these processes, allowing teams to focus on their core work without being bogged down by complexity.

Kasper von Grünberg offered a simple yet powerful definition that captures the essence of a golden path: “Any procedure in the software development lifecycle a user can follow with minimal cognitive load and that drives standardization.” This definition emphasizes the role of Golden Paths in reducing complexity and promoting standardization across the organization.

Golden Paths are often referred to as Abstractions, as both terms convey the ultimate goals of simplification and standardization. As Golden Paths in your platform grow, they form a catalog of services, effectively creating a platform API—a collection of requests that internal users can access in a self-service manner to interact with the platform’s capabilities.

It’s essential to build this collection with the mission of staying relevant, which means considering the evolution of these paths over time.

Golden Paths Design Principles

Google provides a valuable guide on designing Golden Paths, emphasizing key principles to consider:

1. Tackle Cognitive Load Issues

• Reduce cognitive load on the target audience by leveraging abstraction and offering comprehensive getting-started documentation.
• The goal is to simplify the developer experience, making complex tasks more manageable.

2. Integrate with Existing Platforms

• Ensure Golden Paths are compatible with your existing internal developer platforms.
• For example, if a developer creates a new service using a Golden Path, it should automatically integrate with the software catalog managed by the platform team.
• Similarly, if a Golden Path repository includes a Kubernetes Deployment template, that deployment should seamlessly apply to a platform-managed K8S cluster and namespace.

3. Cover the Entire Development Lifecycle

• Illuminate the entire path from development to production.
• This might involve providing instructions for local development, templated CI/CD pipelines, and infrastructure-as-code templates for staging and production environments.

4. Provide Self-Service Capabilities

• Ensure that any developer in the organization can easily discover and use the Golden Paths without the need for a ticketing system.
• Empower developers to leverage these paths independently.

5. Work at the Right Level of Abstraction

• Offer a clear abstraction without concealing the underlying infrastructure.
• Under a DevOps shared responsibility model, developers will need to understand the full stack of their services to maintain, optimize, and debug them effectively.

6. Be Specific and Opinionated

• Tailor Golden Paths to address your organization’s specific needs.
• For example, if your organization primarily handles "brownfield" migrations, design Golden Paths that support these migration paths while being less focused on new "greenfield" services.

7. Maintain Flexibility

• Design Golden Paths to accommodate the needs of multiple audiences.
• For instance, a Golden Path might default to a SQL database but should allow easy substitution with a NoSQL database if required.

8. Keep Adoption Optional (Not Golden Cages)

• Golden Paths should be a resource, not a restriction.
• Developers should have the freedom to choose whether to adopt a Golden Path based on whether it meets their specific needs.

Golden Paths Misconceptions

Platform Engineering is often mistakenly associated only with infrastructure and DevOps simplification, or seen as simply a developer portal. If your organization focuses solely on these aspects, it risks only scratching the surface of Platform Engineering's true potential, missing out on its full business value.

• Platform Engineering as Self-Service Infrastructure Tools: Some organizations concentrate on creating self-service tools that give developers autonomy over infrastructure management.
• Platform Engineering as a Developer Portal: Others focus primarily on improving the developer experience by simplifying coding and deployment processes.
• Platform Engineering as a Marketplace of Reusable Components: Some adopt a marketplace approach, building a repository of reusable components like containers, data, and APIs.

It’s essential to understand Platform Engineering's broader goals and benefits to avoid these misconceptions and unlock the full potential of a platform strategy. A successful platform strategy should go deep, with strong foundations that drive meaningful impact across the organization—not just surface-level improvements.

Contract Model for the Golden Paths

Platform Engineering promotes a contract-based relationship between internal users and the platform:

• Platform Teams: Write and expose Golden Paths as part of the platform API.
• Dev Teams/Internal Users: Request platform capabilities through this API.
• Platform: Provides a control plane to reconcile user needs with the current platform status, ensuring a seamless experience.

How are Golden Paths consumed?

A key aspect of Platform Engineering is providing flexible and intuitive ways for internal users to consume platform capabilities through Golden Paths.
Since different teams and users have varying needs and preferences, the platform should offer multiple methods to access these capabilities, ensuring that they are easily integrated into diverse workflows.

1. Developer Portals

• A developer portal serves as a user-friendly interface that showcases the available platform capabilities, making it easy for teams to discover and consume Golden Paths.
• Through the portal, users can browse a catalog of services, access documentation, and initiate actions with just a few clicks.
• This method is particularly useful for less technical users or for those who prefer a visual interface.

2. Command-Line Interface (CLI)

• The CLI provides a fast, scriptable way to access platform capabilities, making it a powerful option for developers who prefer direct command-based interactions. With the CLI, teams can quickly execute tasks, automate workflows, and access Golden Paths in a way that integrates seamlessly with their existing tools and scripts.
• This method suits more technical users who are comfortable with command-line operations and seek efficiency and control in their workflows.

3. YAML Files

• For teams that favor configuration-as-code, YAML files offer a straightforward way to define the necessary parameters for consuming platform capabilities.
• By providing a simple YAML file with essential configurations, development teams can easily integrate Golden Paths into their CI/CD pipelines.
• This approach is ideal for teams accustomed to managing infrastructure through declarative configurations, offering them a flexible and familiar way to integrate Golden Paths directly into their existing workflows.

Offering multiple ways to consume Golden Paths ensures that the platform meets the diverse needs of its users. Some teams may prioritize speed and simplicity, while others may require more control and customization. By providing different levels of abstraction and interfaces, the platform can cater to a broad range of use cases, driving wider adoption and more effective utilization of platform capabilities.

Enabling GitOps Practices

One of the key advantages of providing multiple consumption methods is the ability to support GitOps practices. GitOps emphasizes declarative configurations and version control, where the entire system's desired state is stored in a Git repository. Teams can use YAML files and Kubernetes CRDs as part of their GitOps workflows, ensuring that changes to platform capabilities are tracked, auditable, and automatically applied to the infrastructure.

With GitOps, teams can integrate Golden Paths directly into their CI/CD pipelines, where changes in the Git repository trigger automated deployments and updates. This approach enhances reliability and repeatability, as all platform interactions are managed through version-controlled code.

By offering flexible consumption methods that align with GitOps principles, the platform enables teams to:

• Maintain Traceability: Every change to platform capabilities is recorded in Git, providing a clear history of who made changes, when, and why.
• Automate Deployments: GitOps automates the application of changes, reducing manual intervention and minimizing errors.
• Increase Security and Compliance: By leveraging Git as the single source of truth, organizations can ensure that all changes are reviewed, approved, and compliant with security policies before being applied.