Velocity and productivity are key for any engineering team, small or large. DevOps practices are key to enabling these metrics. DevOps is about culture, but it also requires the use of tools to become a reality. DevOps ensures that all developers are working in harmony and provides best practices for delivering software efficiently. There is a lot of talk in the software industry about AI/ML usage, and things have gotten pretty interesting since the induction of Large Language Models (LLM), and ChatGPT in particular, into our lives. In this article, we will explore the practical use of AI in DevOps.
First, let’s discuss the evolution of DevOps and where the industry is heading towards from here. Let’s go!
The Evolution of DevOps
DevOps has come a long way, and as a result, we can see one or the other tool popping up every day.
- The Emergence of DevOps (2007):
The term “DevOps” was coined in 2009 by a Belgian software developer, Patrick Debois. It emerged as a response to the growing need for improved collaboration between development and operations teams, driven by the rise of agile methodologies and the demand for faster software delivery.
- Cultural Shift and Automation (2008-2009):
There was still some confusion between Devs and the Ops folks with their tasks, and even though the word ‘DevOps’ was popping up here and there, it wasn’t used as a concrete methodology/practice in organizations. Hence, during this phase, the focus shifted from merely merging teams to fostering a cultural change emphasizing collaboration, shared responsibility, and continuous improvement. Automation played a crucial role, enabling organizations to automate repetitive tasks, code deployment, and infrastructure provisioning. Tools like Puppet and Chef gained popularity, streamlining configuration management.
- Continuous Integration and Continuous Deployment (2009-2010):
Soon, Continuous Integration (CI) and Continuous Deployment (CD) became an important theme for organizations, especially for cloud practitioners, as it helped them find bugs close to the development, shorten the feedback loop and deploy faster than ever.. CI focused on regularly merging developer code changes into a shared repository, and CD aimed at automating the release process to beat the time to market. This is when tools such as Jenkins and Travis CI became popular, enabling faster feedback loop and reducing time to market.
- Containerization and Microservices (2011-2013):
The introduction of microservices enabled the splitting of the humongous monolith applications into smaller pieces to foster easy development, collaboration and deployment. To package these microservices, container technologies pioneered by Docker became a game-changer for DevOps. Containers allowed developers to package applications with their dependencies, enabling consistent deployment across different environments. As a result, microservices architecture gained traction, promoting the development of loosely coupled, independently deployable services. Kubernetes emerged as a powerful orchestration tool for container management.
- DevSecOps and Shift-Left Security (2015):
The broader usage of 3rd party libraries and APIs raised questions around security in their development pipeline. This gave rise to security practices and made security every engineer’s job. Collectively, this security approach was termed as DevSecOps. Integrating security into the entire software development lifecycle became crucial, emphasising “shift-left” security, where security considerations were introduced early in the development process. Security scanning tools, such as Snyk and SonarQube, gained prominence.
- Cloud-Native and Serverless Computing (2015):
DevOps practices were booming, and cloud-native technologies such as Kubernetes emerged to solve the challenges of container management at scale. Even though Kubernetes was introduced years ago, the trend gained momentum only starting in 2018. Companies started ditching Docker Swarm and then using Kubernetes to handle the container orchestration part. Organizations embraced cloud services, leveraging the scalability and flexibility they offered. As companies started migrating to the cloud, the one thought that emerged was why they couldn’t use the services only when required. Serverless computing gained traction, enabling developers to focus on writing code without worrying about the underlying infrastructure. AWS Lambda and Azure Functions were popular serverless platforms.
- AIOps and Observability (2021-present):
The increasing complexity of modern systems led to the rise of AIOps (Artificial Intelligence for IT Operations) and observability practices. AIOps leveraged machine learning algorithms to automate problem detection, analysis, and resolution. Observability focused on gaining insights into system behaviour through metrics, logs, and traces. As a result, tools like Prometheus, Grafana, and ELK stack (Elasticsearch, Logstash, Kibana) gained popularity.
The future: DevOps AI Assistant and AI-based Platform Engineering
The complexity of DevOps with the introduction of Kubernetes, Terraform, Helm Charts and other tools led to increased overhead of DevOps teams on one hand while also increasing the developer dependency on DevOps on the other hand. Organizations couldn’t keep up with the talent debt while a major portion of existing DevOps resource time was focused on addressing developers’ requests- enter the world of Developer Experience in Platform Engineering.
Also, we can’t ignore internal developer portals here as there is already a big buzz around this. Platform engineering and internal developer platforms are indeed emerging as significant trends in the future of DevOps. These approaches aim to streamline and enhance the software development process by providing developers with robust and scalable platforms that facilitate efficient and collaborative work.
In Gartner’s Hype Cycle of 2022, two emerging trends that have gained significant attention are Generative AI and Causal AI. Generative AI refers to the technology that enables machines to produce creative and original content. Causal AI, on the other hand, focuses on understanding the cause-and-effect relationships within complex systems. Both Generative AI and Causal AI represent promising advancements that hold the potential to reshape the way we create and understand information and systems in the future.
Platform engineering & internal developer platforms (IDPs) aim to enrich developer experience through self-service platforms that help develop, deploy, and operate software applications. The main goal here is to provide your developers with consistent environments, robust infrastructure and a standard automation workflow so that they can focus on writing code rather than doing everything themselves from the ground up.
A DevOps AI assistant is an invaluable approach emerging for overwhelmed software development and operations teams alike, helping them easily automate the DevOps tasks such as CI, CD, code scan, configuration management, infrastructure provisioning, monitoring, using natural language and without the complexity of tooling. By integrating with existing DevOps tools and platforms, the virtual assistant can provide real-time insights, notifications, recommendations and actions in a conversational way, thus making DevOps and engineering platforms accessible to everyone and extending DevOps to the rest of the engineering organization. This will improve developer velocity and level up their experience while reducing the organization’s overhead cost and the need for additional headcount. If all of this sounds too good to be true- enter Kubiya
Kubi: Your DevOps Assistant
The emergence of ‘Kubi: Your New DevOps Assistant’ has created a significant buzz in the DevOps space, offering a game-changing solution for teams involved in software development and operations. With its advanced capabilities, Kubi leverages Large Language Models throughout its entire stack, integrating conversational AI into its algorithms where it automates repetitive tasks, provides actionable insights, and facilitates seamless collaboration within DevOps teams. In addition, by integrating with existing DevOps tools and platforms, Kubi streamlines processes such as code deployment, testing, monitoring, knowledge retrieval, and incident management, enabling teams to focus on higher-level strategic activities. The introduction of Kubi marks a paradigm shift in the DevOps landscape, where for the first time, organizations can achieve greater efficiency, agility, innovation and SLAs in their software development lifecycle without needing to add headcount.
How Does Kubi Work?
Kubiya is a ChatGPT-like experience for DevOps as it uses generative AI to create automated workflows that integrate with your Git/CI/K8s/Cloud/other engineering platforms and make them accessible to your users through a conversational AI over your existing chat tools while keeping permissions and TTL.
It uses proprietary large language models to converse with end-users, understand the context of their request, and identify missing information to completely understand the required action. It then uses pre-build action stores wrapped inside of workflows to execute those actions while taking into account permissions.
Creating Extensive Workflows
Create extensive workflows in minutes and share them with your team to reuse and work. Or you can get started with the already available workflow templates.
Kubi integrates with any API or SDK, so, therefore, it’s fully extensible to almost any DevOps tools out in the market, making sure developers’ lives are easier than ever. Extending the integration to other tools, such as homegrown tools, is easy by using a simple Python decorator.
Integration with any DevOps Tools
For example, here are some of the actions supported for AWS
Workflows can be easily customised or tested using their webapp, if you want to put proper guardrails and filter our certain data from your end-users.
Managing Workflows in a Declarative Format
You can also manage your workflows through YAML.
Security is Taken Care
You can easily manage permissions and RBAC so only authorised personnel can have the ability to create and modify the workflows. This way, your security concerns are taken care of easily.
Knowledge Management Q&A Engine
But Kubiya can go beyond just actions. It can learn from your docs (!!) and answer “How do i…” questions based on your Confluence, Notion, Gitbook or any other markdown resource
Repetitive and mundane tasks in DevOps can be daunting and can drain the energy out of your developers. It is time to say goodbye to those complex configuration tools that add a burden to your engineering team. Kubi helps you manage your developers and DevOps engineers’ time so they can do more in less time.
Why Should Software Organizations Use Kubiya?
DevOps self-service, powered by an AI assistant like Kubiya, offers several notable advantages as a peak into the future of software development and operations.
- Zero learning curve to adopt: Work Smarter, not Harder for organizations to truly adopt a scalable DevOps practice the end-user experience needs to be prioritized. In today’s LLM obsessed world, it’s been almost universally accepted that this begins and ends with the presence of ConversationalAI.
- Flexible and easy to maintain workflows: Why limit yourself to the boundaries of rigid rule-based workflows when you operate with prompt-based dynamic ones. Prompting your intent into Kubiya will generate a predictable techstack-aware and permission-aware workflow-less workflows.
- Increased Efficiency: eliminate context-switching and improve upon DevOps self-service without sacrificing critical time to market. By leveraging AI assistant tools like Kubiya, tasks such as code deployment, testing, and monitoring can be streamlined and performed more efficiently by extend actionable insights and workflows i real time, and bypassing the ticket queue.
- Enhanced Collaboration: Kubiya acts as a virtual teammate, assisting DevOps teams throughout the software development lifecycle. It promotes stronger teamwork by fostering better communication and knowledge sharing.
- Continuous Delivery and Integration: Kubiya can help automate continuous integration and delivery (CI/CD) pipelines. It can handle tasks such as code merging, unit testing, and automated deployments, ensuring a smooth and reliable release process. This enables organizations to deliver software updates rapidly and frequently, supporting agile development practices.
- Intelligent Automation: Kubiya leverages LLMs as a means to gain user intent, and gather the necessary context to extend actionable workflows complete with reinforcement learning that helps fine-tune the AI assistant response and serves a more personalised recommendation the next time around.. These techniques enables users to understand and respond to user queries, automate repetitive tasks, and provide intelligent recommendations. Through intelligent automation, DevOps self-service becomes more intuitive, reduces manual effort, and increases productivity.
- Continuous Monitoring and Analysis: Kubiya integrated into DevOps self-service workflows can continuously monitor systems and applications, providing real-time insights and alerts. It can analyze logs, metrics, and other monitoring data, identifying potential issues or performance bottlenecks. By proactively detecting and addressing problems, DevOps teams can maintain high system availability and performance.
- Level Up Non-Technical Users: Kubiya empowers non-technical users to interact with the development process more effectively. It can guide users through complex tasks, offer recommendations, and automate repetitive processes. This democratization of DevOps functions allows stakeholders from various roles and departments to participate actively in the software development lifecycle.
- Knowledge Capture and Retention: Democratize access to knowledge with Kubiya by capturing and retaining knowledge from interactions with DevOps teams, internal docs, data sources (eg Jira, ServiceNow, etc) and user interactions. By continuously learning from knowledge sources, operator inputs and end-user responses, the assistant becomes more intelligent and efficient over time. This knowledge can be shared across the organization, ensuring continuity and reducing dependency on specific individuals.
Why wait when you can automate your DevOps today?
Last week we had our first Engineering Leaders Forum, here in the Bay Area. Over 100 VPs and SVPs of Engineering from the Bay Area’s leading companies registered. At some point we had to open up a waiting list. Good problem to have.
I have wanted to hold such an event for quite some time. Engineering leaders, as opposed to other personas, lack the opportunity to share their challenges and hear the perspective of others in a very safe environment. In the room last week we had a few thousands of years of experience. Based on the feedback, the format of roundtables was well received. I really appreciate the fact that both Kubiya.ai as well as Devzero, Turing, Uplevel and AWS chose to sponsor the event and make it happen. I want to talk a little bit about some of the insights that we had from executives in the room. Again, I’ll keep it generic so that I wouldn’t share anything confidential.
Two key items that we heard from all the executives in the room, one was centered around AI and the other one was around measurement.
What do Engineering leaders think about AI and ChatGPT?
The feeling is that we’re in an interesting era. The same transformation that happened 10, 15 years ago with the cloud is happening today with Artificial Intelligence (AI). Everyone wants to “do something” around AI. I also hear that from our customers and prospects. But many are still figuring out what exactly they want to do around AI and what are the use cases. We heard lots of concerns about data privacy. Members mentioned that with tools like ChatGPT, they don’t really know how data is stored, what is stored and who might have access to it. One mentions that the same way his company can gain an edge by using AI, he can also give an edge to their competitors using their data. Most leaders are taking baby steps towards using AI. A few companies in the room mentioned actually trying various things around AI, using Copilot or ChatGPT, to generate code for example.
The other thing that we heard very strongly and we ourselves are in the midst of it so we weren’t surprised with that, was the fact that AI today is very generic and lacks the contextual organization. And so as opposed to giving generic code snippets, if AI could look into the organizational repo and make more contextual recommendations, or the case of Kubiya, if AI could actually figure out the engineering resources and platforms that an organization is using, and be able to spin up environments based on that, or render Terraform models, that will be very powerful.
Early adopters mentioned that they’re using AI mostly for code reviews. Developers find code reviews to be unproductive, so having another set of “eyes” can expedite the process and remove dependencies.
Again, in some ways it feels like AI is a disruptor and everybody wants to be in the game and use it to gain an edge, but the use cases are still not there and data privacy is a big issue.
One additional item that was discussed was the use of ChatGPT inside organizations. On the one hand, unless they block the domain, organizations have no control over how people in the organization use ChatGPT. You can come up with policies to restrict the use of it but it’s difficult to enforce that. If one uses ChatGPT to write code or create a document you are exposed to sharing private information or violating licensing rights. Organizations don’t have great policies right now nor giving clarity to employees around when to use AI and ChatGPT. One last thing that was mentioned specifically around tools like Copilot and other AI code generating tools is the fact that given that code snippets were used to train their LMM, the output at some point might violate copyrights and will expose them to legal actions.
How do engineering leaders measure their teams?
We had a number of roundtables on scaling engineering organizations and increasing velocity. My colleague Debo Ray from Devzero wrote about it in “Impact of AI, Product Velocity, and more: Learnings from Engineering Leaders Forum”. At the end of the day the discussions quickly came back to the theme of how do you measure engineering organizations? Some of the leaders in the room weren’t as familiar with DORA as others. And so it really feels like managing and measuring an engineering organization is still as much as art as it is science. One of the leaders mentioned around the fact that he’s looking at the number of commits per week, per developer as a measurement, and that together with some context around whether that engineer was sick, busy in interviews, in a conference or doing something else, over time gives them good visibility to how the team is performing.
Remote work and RIFs
Prior to the roundtables we had a chance to talk to members 1-on-1. It quickly came down to two topics. Layoffs and remote work. Many of the leaders had to reorganize their teams and let go over good people. Leaders these days are more focused on efficiency than scale.
It was interesting to hear the views around remote work. 12-18 months ago companies were talking about remote work being the new norm. Most members were scaling back the transition to remote work, citing the overhead of managing remote teams, lack of visibility and the impact on culture as being the key reasons they ask developers to work from the office 2-3 days a week. Especially with layoffs, culture and communication becomes important and it’s easier to communicate and build culture with teams being onsite. Remote work also seems to impact engineering leaders themselves. Working remote from your home office seems less favorite than it used to be. We work longer hours, don’t socialize enough and lack better work-life balance.
Members really enjoyed the format and the opportunity to share knowledge and experience with one another. The discussions were frank and transparent and the combination of in-person 1-on-1 conversations with a 10-12 ppl roundtable offered lots of insights and food for thought.
See you on the next ELF.