Platform as a Product: Scaling DevOps with IDPs.

If there is something that characterizes the world of IT, it’s the constant change that surrounds those of us who work in it. Years ago, we started to popularize the term DevOps, which finally gave a name to a large group of professionals who, in their daily work are capable of combining development and operations into a single, self-sufficient profile to deliver software.

However, our world never stops changing, opening up a wide range of tools that grows every day. Having so many options and required areas of knowledge means that the cognitive load on development teams increases exponentially.

This complexity led to the emergence of the Platform Engineering philosophy, created to promote a mindset shift where the infrastructure team treats their work as a product that (among other benefits) enhances the developer experience.

This is how the IDP or Internal Developer Platform was born as one of the key outcomes of the Platform Engineering discipline. In this post, you’ll learn why this term is important and how it can help DevOps culture scale to meet market demands.

What does IDP mean?

Let’s start with the basics. IDP stands for Internal Developer Platform, and if you’ve never heard of it, don’t worry. Three years ago, I didn’t know what it was either.

As our colleagues David Morales and Alejandro Cavero explained in their post “Key Elements of Platform Engineering”, the IDP is the cornerstone of Platform Engineering. An IDP is an architecture composed of tools and services offered by a Platform Engineering team with a “Platform as a Product” approach.

The goal of an IDP is fairly simple: to allow developers to focus solely on writing software, leaving the platform to handle operations and deployments securely and following quality standards.

What does an IDP have to do with Platform Engineering?

We could say that IDPs emerged as a technological response to the needs identified by Platform Engineering, but in reality, the IDP was born as an answer to a set of questions:

• What problem led to the Platform Engineering philosophy?

Platform Engineering emerged because things got out of hand with the number of tools developers needed to know to get their work done.

For example, today it’s common for any respectable DevOps professional to know at least one container orchestrator (Kubernetes, Docker Swarm, ECS…), a Kubernetes-based platform (OpenShift, Rancher, Tanzu…), some IaC tool (Terraform, CloudFormation, ARM, CDK…), and a CI/CD system (Jenkins, ArgoCD, Tekton, GitLab CI…).

Instead of focusing exclusively on writing code, developers need to understand an entire tool stack just to perform their jobs, which represents a serious cognitive burden that takes time away from what should really matter: development. It’s precisely to solve this cognitive load problem that the Platform Engineering philosophy began.

• What would be the ideal solution to the cognitive load problem?

The ideal solution is surprisingly simple: a mindset shift. If infrastructure is too complex for development teams to master, then why not offer a Swiss-army-knife product that gives them everything they need? At this point we know what we need—but how do we implement it?

• How can we provide development teams with the tools they need without overwhelming them?

This is where the IDP makes its stellar entrance, turning the idea (Platform Engineering) into a tangible product available to teams.

An IDP is the software layer that encapsulates and abstracts all the complexity of the tools development teams use. Instead of interacting with ten different tools, developers interact with a single, simple IDP interface to achieve the same results securely and in a standardized way.

So far, we’ve covered the motivations behind the IDP as a key concept for development efficiency—but we still haven’t defined some key components of an IDP.

Key components of an IDP

A well-built IDP has four components that truly matter.

First, a solid developer portal (usually Backstage) that acts as the single entry point. If you’ve never used it, I recommend checking out their website to see everything it can do.

Second, the tools offered by the platform. And here comes the debate: paid but reliable tools (COTS) or open-source ones? At this point (and depending heavily on what you want your IDP to be), you must make decisions like “Should I pay for Datadog or maintain Prometheus myself?” (Spoiler: there’s no right answer. You must assess your needs.)

Third, closely related to the deployed tools, is IaC (Infrastructure as Code). All resources and tools in the platform must follow the same principles of automation and resilience (we don’t want surprises during Disaster Recovery). For this reason, resources are usually deployed using modules and templates from IaC tools like Terraform or configuration management tools like Ansible.

And last but not least, CI/CD templates that let development teams only think about questions like “What do I need to deploy today: containerized microservices or a serverless function?” and let the platform handle the rest.

How can an IDP bring value to your company?

So far, everything sounds great, right? We’ve seen advantages in security, quality, and reducing the burden on development teams. But what additional value can an IDP really bring to an organization?

• Deliver more, faster: or, in marketing terms, “reduce time-to-market”. The IDP includes an updated catalog of microservices, libraries, and components configured from the platform (scaffolding), helping teams work more efficiently.
• Reduced cognitive load: honestly, if I had to choose a single key point for an IDP, this would be it. Developers no longer need to master specific technologies; the platform provides them through the developer portal. Quality and security teams also rest easy because the IDP enforces configurations that guarantee secure, compliant deployments.
• Governance by default: if you’re a security professional, you know the struggle of getting teams to adopt security policies. An IDP eliminates that problem, including by default the compliance, security, and hardening policies configured by the operations team.
• A single entry point for operational and development management: so far we’ve talked about the benefits for developers, but an IDP can also bring typical operations tools together into a single, user-friendly interface. No more watching 20 dashboards to figure out what broke this time!

In summary, the benefits for development are huge, but where do the operations teams fit into the IDP picture? Are they just providers?

The role of the operations team in an IDP

We’ve been talking nonstop about how an IDP is fully oriented toward simplifying the life of developers, but where does that leave the operations team?

The operations team—or, sticking to IDP terminology, the Platform Engineering team—is critical to ensuring that an IDP is truly aligned with project needs.

Their work is not limited to deploying infrastructure and configuring the platform; it also involves collecting requirements directly from platform users (developers) to continuously optimize the IDP, making it faster, more efficient, and more reliable.

In fact, an IDP often includes tools like Backstage (mentioned earlier), which not only serve as a frontend for developers but also act as a central observability hub for operations, aggregating metrics, logs, and service health in one place.

When you think about it, viewed this way, Platform Engineering teams get closer to the role of product development teams—understanding and responding to user requirements—bringing infrastructure and development closer together again, just like when we talked about DevOps at the beginning of this post.

Conclusions

After years of seeing brilliant teams waste time fighting with Kubernetes YAML and permissions, I can confidently say that IDPs are not a trend—they’re here to stay.

IDPs are the natural evolution of DevOps as organizations grow. That said, don’t try to implement one if your team has fewer than 50 developers. Although IDPs are designed to reduce complexity, a poorly designed IDP can add even more complexity than you already have.

References

If you found this post interesting, I recommend checking out the work of my colleagues—true Platform Engineering rockstars:

• Key elements of Platform Engineering
• How to start with Platform Engineering: team, kickstart, roadmap
• Measuring Success in Platform Engineering
• How to Release a Platform as a Product?