Internal Developer Platforms and Conversational AI

There is plenty of conversation about ‘platform engineering’, ‘self-service developer platforms’, ‘internal development platforms’ and ‘DevOps self-serve’ today. This is because the cloud-native ecosystem has realized that CI/CD pipelines are not enough to handle the complexity of Kubernetes infrastructure and the teams that run applications on it. The rise of the internal developer platform (IDP) has brought with it many opportunities but also challenges that can’t be ignored. In this article, we look at the promise of an IDP, and the possible options when building an IDP.

What is an internal developer platform?

In simple terms, an internal developer platform is a self-service platform that developers can use which eliminates dependence on Ops teams to provision and deploy applications. The internal developer platform essentially features a set of tools, techniques, APIs, and other support services, with which developers can be self-reliant.

In terms of setting up, your operations team designs this self-serve layer to ensure your development team can create (on their own) the required infrastructure resources based on their applications’ needs. One of the main benefits of the IDP is that developers can release products and features faster. 

The Ops team will need to provide detailed configuration and templates for the most commonly used resources, along with defining roles and permissions to ensure security for these resources. For example, they could create a template that spins up 3 EC2 instances on AWS and an S3 storage bucket along with 5 Lambda services, all talking to each other through AWS App Mesh, and being monitored by CloudWatch and sending logs to CloudTrail. Now, provisioning all these AWS services repeatedly for 20 developers at different times is a lot of duplicate work for the Ops team. In the process, developers end up waiting for days or weeks, and the Ops team experiences burnout. To counter this, the internal development platform empowers developers to create these resources themselves on-demand, while Ops can rest assured that the resources are approved by them. 

With self-service platforms eliminating the complexities of infrastructure configuration and orchestration through an automated process, they ultimately boost productivity levels and enhance the quality of work of both developers and Ops teams. As time-draining and repetitive tasks are taken out of the loop, developers can spend more time writing code, and Ops teams can focus on more strategic tasks for the business.

Build or buy an internal developer platform?

Building an IDP in-house is not for the faint of heart. It involves intricate planning and preparation to understand the needs of the various development teams, defining the architecture design of the platform, getting buy-in from various stakeholders - and all this is before the platform is even built. 

The building of the platform itself requires collaboration between various members from both the Dev and Ops teams. There are open source options like Crossplane available that have a Kubernetes-centric architecture and are promising, but they come with their own learning curve, and no hand holding along the way.

Commercial options to build an IDP offer a more clear path, and support that’s readily available (although building out workflows and integrations can be complex). However, the main problem with this option is the possibility of vendor lock-in. Especially if the vendor offers custom plug-ins and integrations that are created by them, it’s possible that these components are not as customizable for every need. Some of these platforms are easy to get started with but later become difficult to manage as usage scales into thousands of clusters and hybrid cloud setups.

There is however, a third path to an IDP - conversational AI. 

The role of conversational AI in an internal developer platform

Although the platform (ops) team provides pre-configured resources and templates for developers to accelerate their application releases, more often than not they will need unique workflows. During such instances, they will have to reach out to the Ops team to spin up the resources based on the specific use case. However, the key aspect here is being able to describe your specifications accurately.

In most cases, enterprises use chatbots to automate this process. But the challenge with chatbots is that the prompts need to be 100% accurate. A lack of context in the developer's request could demand a human (read operations team) to intervene to facilitate the requirement. At this stage, there is a back-and-forth to understand the ask better and provide the infrastructure deemed necessary. This puts an added burden on the operations team, which defeats the entire idea of having a self-service layer.

While a baseline chatbot couldn’t handle such ambiguous requests from developers, a conversational AI-powered virtual assistant can be a great fit here. It can use simple conversations to gather context if it is missing in the original request made. The AI-based virtual assistant can decode the intent from even fragmented or unclear phrases using LLMs (large language models) and offer a relevant resolution. 

You can implement an intent-based conversational AI at the team or enterprise level considering your security and governance policies. It allows developers the freedom to innovate faster by making available new workflows whenever they need them.

Let’s look at Kubiya, the conversational AI virtual assistant we’ve built to greatly simplify how DevOps self-serve can be implemented and provide you with an easy-to-use and flexible internal developer platform.

Kubiya’s approach to internal developer platforms

Think of Kubiya as your AI-powered DevOps virtual assistant that can get things done for you without you having to go through all the steps manually. With loads of out-of-the-box integrations and an SDK that allows you to extend Kubiya to your entire tech stack, the lift up is effortless. With local runners you can use Kubiya for any tool or platform you use without having to share permissions or access with Kubiya…and no worries about vendor lock-in as there is no limit to what you can use Kubiya for.

You can create workflows in seconds using a simple English prompt and then fine-tune the different steps and actions using our no-code builder, or if you prefer our DSL (domain specific language). Once you’re happy, you can publish the workflow and have it ready to be consumed by developers.

In terms of security, Kubiya connects with your identity provider so you can define workflow and resource access policies for all users in a few clicks. 

For developers, they can interact with workflows directly from Slack - a platform they’re already familiar with - so no learning curve or context switching there which is great for adoption. They can also use CLI if they prefer. Support for Microsoft Teams is coming soon. 

Kubiya’s AI capabilities shine here as well as it is able to understand the exact needs of the developer and provision resources from workflows that have been pre-created by the platform team. 

Kubiya is the next step in the evolution of the internal development platform. 

Conclusion

An internal developer platform gives developers the autonomy to deploy applications without depending on the ops or platform team. However, the inherent challenge for any organization here is how to build and manage this IDP. Open source options are risky, and vendor platforms also require a heavy lift-up and also can result in lock-in.This is where Kubiya adds significant value. Kubiya leverages the power of AI to make the job of both platform and development teams easier. It performs tasks that humans are required to do in seconds or minutes rather than hours. Yet, it gives the platform team the peace of mind knowing they have full control in what gets provisioned, by who, and where it is provisioned. Give Kubiya a try, you’ll never think of an IDP the same way again.