Why we need IDPs

Table of content:

1. Why we need IDPs
2. The building blocks of an IDP

‍

DevOps, the set of practices, tools and cultural philosophy that automates and integrates the processes between software development and IT teams, has completely revolutionized the modern software delivery paradigm over the last 15 years. From its onset in 2007, DevOps has grown to be the predominant method for the software delivery lifecycle. GitLab’s 2023 Global DevSecOps report found that 56% of respondents reported using DevOps or DevSecOps methodologies (source) and market research shows the market value for DevOps is 10.4B USD in 2023 with projected growth to 25.5B in 2028 (source). 

‍

While DevOps has profound benefits including faster software delivery, more frequent releases, increased flexibility, greater scalability and availability, better resource utilization and more, it has brought its own set of challenges. As operations shifted left, developers became increasingly burdened with a multitude of tools, managing infrastructure, and participating in workflows that would typically fall to other types of engineers - rather than writing code. In addition to the new job tasks developers face as part of the “You built it, you run it” approach, cloud technology became more prevalent ushering in yet a new era - the cloud native era.  With the cloud-native era well underway, developers needed to learn more complex cloud native technologies such as Kubernetes. These new technologies not only added cognitive load, but slowed down the developer process with repetitive manual tasks for cloud resource management. The advancements didn't stop there - shift left security, microservices architecture, infrastructure Code (IaC) and GitOps - all piled on developers and DevOps managers plates. DevOps managers and engineering leaders began to realize that as the demand on developers increased, the benefits of DevOps were being diminished by resourcing bottlenecks, manual repetitive tasks (DevOps toil) context switching, cloud resource management and more. The evolution of DevOps has created pain points across the engineering organization.

‍

Today's challenges for engineering functions

‍

As engineering organizations looked to find ways to ease the burden of the modern tech stack on DevOps Managers and developers  themselves, Platform Engineering gained a presence as a full-fledged discipline. Platform engineering is the “discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era.” Platform engineering is focused on supporting developers so they can focus on code, not wrangling the tools they need to write, build, test, secure, deploy and monitor the code. Accordingly to Paul Delory, VP Analyst at Gartner. “Platform engineering emerged in response to the increasing complexity of modern software architectures”. Platform engineering was recognized in top ten tech trends for 2023 by Gartner and included it in The Gartner Hype Cycle. 

‍

Gartner expects that by 2026, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery.

‍

As the discipline of Platform Engineering grew in popularity, Platform Teams and Internal Developer Portals (IDP) surfaced as ways to implement the different approaches to easing the tech stack burden. Both have shared goals of equipping developers with the tools and infrastructure they need, while stripping away the complexities. This setup boosts developer productivity by keeping their focus on code, not tool wrangling. The outcome of either is a set of “golden paths”, as defined by Spotify “a set of opinionated and supported paths to build something”. In the next Chapter/blog post, we’ll dig into how IDPs are solving for DevOps toil and becoming more critical than ever. 

‍

Puppet’s 2023 State of DevOps: Platform Engineering Edition showcases the benefits – 94% of respondents are in widespread agreement that the adoption of Platform Engineering is helping their organization release the benefits of DevOps.

‍

‍