DevOps is a portmanteau of development and operations. Simple definition, blog post over.
Of course, in practice, DevOps is much more nuanced. The signal there is in practice — while there are different schools of thought on whether DevOps is a set of tools or a culture or a process, one thing that is consistent is that it is a thing that is done. It is not a theory (which agile development kind of is) and it’s not a tooling change in that simply changing tools won’t change your outcomes. It really is a way of practicing how your applications are designed and delivered.
The Background: Agile and CI / CD
DevOps means a lot of things to a lot of people, and its meaning depends on the context. Because it is a rather vague term, DevOps is frequently confused with a couple of other critical concepts, namely agile development and continuous integration / continuous delivery (CI/CD). The associations are fair — it’s just important to understand what those associations are.
The first foundational concept is agile development. In a terrific piece on DevOps as a concept, Ernest Mueller of the Agile Admin blog calls agile the older brother to DevOps. Agile is very much rooted in concept and culture; it is a state of mind. The core of agile is a series of values, defined in the Agile Manifesto, that outline an ideal state of software development. These values are then refined into guiding principles and further refined into methods that can be used by teams implementing an agile environment.
- People over process over tooling
- The goal is to create software that (mostly) people can use without docs.
- Customers are your collaborators.
- Responding to change is more important than following a plan.
The thing is, those principles don’t necessarily change any processes or tools — they first influence your perspective.
Agile codified the understanding that there are a lot more stakeholders in software development than developers alone. Customers, QA, business executives, UX, and documentation all have a vested interest in how software is designed and implemented, and they can all contribute to that overall development process. It’s people over process — the first rule of agile is collaboration.
Once that perspective changes, then you’re on to associated concept #2, CI/CD. Agile development has a purpose: the rapid release of an application. So, when that is the perspective, the result is a working system which results in continuous delivery. The mantra here is “build, deploy, test.” The continuous integration part of that is the backend, the automated processes that take a code commit, send it to a build system, and send the build artifact to a QA environment.
CI/CD is frequently related with agile because that continual, automatic release process fits nicely with a rapid release mindset, although they are different concepts. CI/CD is where the more pragmatic concerns about tooling, infrastructure, and organizational ownership begin to creep in.
Agile defines a software development process, and that process ends with code complete. In the real world, while you can silo the developers so that code complete is the finish line, you can’t silo the code. That application has to be deployed to a production system and then maintained, through all manner of user interactions. That’s the purview of the IT operations team.
In a sense, the term DevOps is a way of moving the goal posts on “done” from “code complete” to “code successfully delivered.”
The collaboration in this sense is to get a much better, shared understanding of what an application is and what it is to be used for. Software operates in a specific environment and every organization has its own unique sets of constraints like hardware quirks and security requirements and network configuration. When development is unaware of the final production environment, they can create software that doesn’t work in real life even if it runs fine in a build system. Likewise, when operations doesn’t understand the design of the software or its intent, then they may make operational changes that harm the application.
DevOps brings the operations team into the development cycle and the development team into the maintenance cycle. If dev knows ops, they design and develop better code; if ops knows the design and purpose of the code, they can deploy and maintain it better.
Middleware, App Development, and DevOps
So, tracing the family tree of agile and devops is very nifty, but where does middleware fit into this?
Let’s reiterate why DevOps is increasingly popular: because it is becoming a business imperative to release software frequently, to even provide applications and services on demand to customers. The entire application lifecycle is shrinking dramatically — an IDC report shows that while 63% of companies release twice a year or less today (as of May 2015, when the report was conducted), by 2017, 69% will be releasing monthly or quarterly. That’s an increase of over 10 times the releases that will be done annually.
The reason for that incredible increase is because of the larger technology trends affecting every industry — microservices and service-oriented architectures, the Internet of Things, and mobile applications. The web app giants like Amazon and Google release multiple times per day, but the requirements to deliver software quickly have expanded beyond web operations.
DevOps is a practical application of agile principles. You cannot create a DevOps environment through tooling, but you support that environment through tooling, and that is where middleware helps.
In a really early post defining DevOps, Stephen Nelson-Smith identified four core problems in traditional development / operational cultures: a fear of change (because environments break); risky deployments (because they’re not certain code will work in production); developing code in an entirely different environment than production; and a lack of collaboration. Three of those four come down to a lack of consistency and knowledge of different operating environments within the same organization. That is a problem that can be addressed with the right tools. As an example, but not limited to:
- Using containers to develop and test code in the exact same environment as production
- Using orchestration and provisioning tools to automate moving code between environments
- Using configuration management to consistently apply changes
- Monitoring all services and systems
Middleware does not define or enforce a DevOps environment, but it can help strengthen the processes that you put in place.