On the way from Continuous Integration to Continuous Delivery
Not every project starts from scratch and can implement Continuous Delivery according to the textbook. It is therefore important to understand the ideas behind the principles of Continuous Delivery in order to be able to answer for oneself which questions companies need to deal with when introducing Continuous Delivery. This article will highlight individual aspects that are particularly important for the journey from Continuous Integration to Continuous Delivery. These include questions such as: Which topics should companies deal with in this context and which technical options are available for a concrete implementation?
The use of a Continuous Integration system has become standard in most companies today. After changes to the source code, the developer receives a minimum level of security with the execution of automated tests by the Continuous Integration system and the feedback of the test results.
After a commit to the source control system, a new build is automatically triggered. The automated tests are then executed and at the end of the build, the developer receives a report on the test results. The results of the tests are then incorporated into the development process. Continuous Integration thus provides rapid feedback in the event of source code changes. If you apply the principle “Integrate early and often” and check in your source code changes at least once a day, you can sustainably improve software quality with this procedure.
Differentiation of Continuous Integration from Continuous Delivery
The Continuous Integration process is completed after the changes to the source code and the execution of the tests and then starts again from the beginning. Continuous Delivery starts at this point and extends the feedback cycle into production. Only when the application has been deployed to production and is available to the customer is the “Definition of Done” fulfilled. Against this background, Continuous Delivery is also referred to as the final stage or “last mile” of Continuous Integration.
Goals of Continuous Delivery
The original goal of Continuous Delivery is already established in the agile manifesto. There, the first principle states, “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” Continuous Delivery addresses this goal by continuing agile development practices into production.
Continuous Delivery is essentially nothing more than a collection of techniques, processes, and tools that improve the process of software delivery. The focus is on the tools and the delivery process. The delivery process can in turn be differentiated under temporal and qualitative aspects.
With the temporal aspect, one tries to accelerate the delivery process, in order to reduce the Time to Market. Management in particular, and increasingly also the business departments, want to place new products or changes on the market more frequently.
In addition to the time aspect, however, the qualitative aspect also plays a significant role. With a rather conservative release cycle of several weeks or months, problems and errors usually occur in the delivery process. This is due to the fact that many manual steps have to be performed by individual persons and the necessary activities can change between the individual delivery dates. Continuous Delivery tries to establish a repeatable and reliable process here predominantly by automating the individual manual steps.
Continuous Delivery Principles and Practices
Jez Humble and David Farley first address principles and practices in the context of Continuous Delivery in their book “Continuous Delivery”. By principles and practices, they essentially mean concrete guiding principles or recommendations that should be followed when applying Continuous Delivery. The article by James Betteley “8 Principles of Continuous Delivery” provides a good overview of the most important principles. If these principles are supplemented with a few more and grouped according to content, the main topics are formed.
The assignment to the topics is not always clear and selective so that no claim to completeness is made at this point. The grouping provides an overview of the topics that companies must deal with when introducing Continuous Delivery.
Key topics in the Continuous Delivery environment
These key topics play an essential role in the principles of Continuous Delivery:
Automation
The most important topic in the context of Continuous Delivery is automation. A repeatable and reliable release and deployment process can only be achieved if many manual steps are automated. Manual steps, which may only be documented in a checklist, are automated with the help of tools and shell scripts in such a way that ultimately only a “button click” is required to trigger the deployment of the application to the target system.
To avoid unpleasant surprises when deploying to different environments, it is important to keep all environments as similar as possible. Meanwhile, there are good tools to set up the environment itself in an automated way (e.g. Ansible, Chef, Puppet, Saltstack). Compared to using simple shell scripts, these tools have the advantage that the execution of the installation instructions is done idempotently. In a mostly declarative manner, the target state is defined (e.g., “package X must be installed and service Y must be in the “started” state”) and the tool then figures out what actions must be performed on the system to achieve the target state. This approach is also known as “Infrastructure as Code” because the definition of the installation instructions is similar to that of traditional programming.
Deployment-Pipeline
The “glue” of the individual automation steps is represented by the so-called deployment pipeline. All steps that are necessary from a single commit to the deployment of the application in production are ensured by the pipeline.
The transition from one stage to the next is represented by the so-called “quality gate”. As soon as an error occurs in the previous stage, the entire pipeline build is aborted at this point (continuous delivery practice: “If anything fails, stop the line”). This ensures that no faulty artifacts make it into production. And not only that: with each passing of the further stages, the delivery artifact also reaches an ever-higher level of quality.
The sequence of test execution is based on Martin Fowler’s test pyramid. Here, too, the goal is to access feedback as early as possible in the event of an error. According to their execution speed, unit and integration tests are executed in the commit stage. Tests of longer duration (e.g. acceptance, capacity, and load tests) follow in the other stages. The pipeline enables parallel processes, which shortens the overall cycle time.
Above all, the pipeline transparently illustrates the entire release and deployment process. With the automation and chaining of the individual steps, one is able to deliver the application to production much more frequently. Apart from the fact that the changes are available to the customer earlier, this approach reduces the number of changes per deployment due to the more frequent deliveries. This also reduces the risk of potential errors per delivery. And if an error does occur, a bug fix release can easily be pushed forward (roll forward) and there is no need to try to get the old version running again (roll backward).
For the realization of a deployment pipeline, the widely used continuous integration server Jenkins is a good choice with corresponding plugins or the continuous delivery server GO from Thoughtworks, which is now also published as open-source. The latter offers significantly more abstraction levels for the definition of the individual work steps and is a good choice, especially for more complex pipelines. Commercial providers have also integrated Continuous Delivery functions into their CI systems (e.g. Bamboo, or Teamcity).
Configuration Management
Configuration management deals with the question of what changes need to be made to an application in order to integrate development into a Continuous Delivery environment as easily as possible.
Firstly, it should be ensured that only one build artifact is created for all environments (DEV, TEST, PROD). The respective active environment is then passed “from outside” when the application is started, e.g. as a JVM parameter. The application then knows which environment-specific settings it should load. In the Spring framework, for example, this is solved using so-called Spring profiles.
To simplify the transport to the target system, the application should place as few requirements as possible on its environment. This can be achieved by bringing its environment itself and rolling it out together with its runtime environment. For example, in the case of a web application with Spring Boot, the entire application including its dependencies and a servlet container is packaged in a JAR file (fat jar or Uber jar). Transporting it to the target system is then nothing more than copying a file. It becomes even easier if you package this file in a Docker container. In the deployment pipeline, the image is created and uploaded to a repository (Docker registry). It can then be downloaded and launched again from the target system. Only Docker needs to be installed on the target system. All decisions about further preconditions (e.g. Java version, operating system patches, etc.) are already made in the Docker image. Alternatively, however, it is also conceivable to use package managers close to the operating system, which package the application in an RPM or DEB package, for example. Since operating staff are usually well acquainted with these tools, frictional losses during commissioning can be avoided.
In the best case, each commit results in a deliverable artifact at the end of the deployment pipeline. For software versioning, the artifact needs a fixed version number. This is because only with a fixed version number can the reference to the respective software status be established. Usually, the pipeline build number is included as part of the version number and tagged in the source code management. In this way, a reference can be made in the event of an error in production.
To ensure that an executable software version is created at the end of a deployment pipeline, the application must be in a deliverable state at all times. With the application constantly evolving, this is not so easy. Unfinished features should not cause any side effects on the rest of the system. This can be remedied by so-called feature toggles or feature flags, which make it possible to selectively switch these parts on or off. This keeps the application in a deployable state and disables unfinished features. Another positive side effect of using feature toggles is that, if desired, new features are initially made available only to a limited range of users. An example of tool-supported handling of these flags from the Java area is a toggle.
Changes to the application may also result in changes to the database or database schema used. When deploying the application, it must be checked whether the database is in the desired target state. Again, tools such as Flyway, Liquibase, or Orcas can help to perform schema migration automatically. It becomes somewhat easier with the use of a document-oriented database that does not require a fixed data schema (e.g. MongoDB, etc.).
Self-Service Operations
In a Continuous Delivery environment, release and deployment are only “one button away”. But where is this button located? This is where self-service operations systems come into play, allowing these automated steps to be unified under one interface. Typical features include the execution of manually or automatically triggered tasks (locally or remotely), rights management, and the history of executed activities. The functions are usually united via a dashboard and enable the user to quickly determine who deployed which version of the software and when. In addition to release and deployment management, tasks such as restarting an application server or integrating smoke tests can also be implemented. Examples of this type of tool are Rundeck or Ansible Tower.
If you do not want to use a separate tool, you can also perform most of these tasks in the deployment pipeline (e.g., by manually triggering a pipeline stage).
Log management and monitoring
Feedback about the running application in production is not only of interest to the operations staff but also provides important information for development. For this purpose, it is necessary that log information is available to every developer via a suitable portal. Due to the increasing virtualization and “containerization” and the associated horizontal scaling, more and more log information is created on different servers, for which there must be central access and a central repository. Well-known representatives of this type of system are the so-called ELK stack (Elasticsearch, Logstash, Kibana) or Graylog. With these, it becomes possible not only to monitor productive logs but also to search them (indexing) and to correlate different types of log information (e.g., also from routers, load balancers, Syslog, etc.). The information obtained can be displayed in an aggregated form in a dashboard or can also trigger automatic notifications if, for example, a certain amount of errors have occurred in a defined period of time. However, for easy processing of log information, it is important that it is sent to the log management system in a standardized format. The Graylog Extended Log Format (GELF) is an example of such a unified format. The logging framework Log4J, which is frequently used in the Java area, offers a ready-made GELF appender as of version 2 for sending log events in this format.
DevOps
Compared to Continuous Delivery, DevOps is more of a philosophy or culture within the company. As a made-up word from Development and Operations, the relationship between these two departments, which is fraught with conflict, is highlighted here. If you take a closer look at the objectives of Development and Operations, it does not seem surprising that conflicts can arise between the two departments. After all, Development has to take care of implementing as many requirements and thus also changes as possible, whereas Operations primarily has the stability of the system as its goal.
It is precisely this rather conflicting objective that shows that the interests of a single department must not be paramount in software development, but that the focus on the entire system defines the goals. Everyone directly or indirectly involved in the delivery process is equally responsible for a successful go-live. DevOps thus also involves other departments such as QA, specialist departments, etc. In the context of microservices, cross-sectional teams are formed that are completely responsible for a complex of topics. In accordance with the principle “You build it, you run it”, overall responsibility is shifted to one team, thus avoiding frictional losses at departmental boundaries.
A helpful step in the direction of interdisciplinary teams can be the participation of an operations employee in development (or vice versa). This simplifies coordination and coordination for upcoming product releases due to the physical proximity to the development team.
Conclusion
Continuous Delivery is the next logical step to support the process of software delivery into production as a continuation of Continuous Integration. The principles and practices of Continuous Delivery address important issues that must be considered on the path from Continuous Integration to Continuous Delivery. The focus is on automating manual execution steps.
For most companies, the frequency of software delivery will certainly not be the sole target criterion, but rather reliable and error-free delivery. From this point of view, Continuous Delivery becomes important for any company doing software development. Unfortunately, there is not one and only tool that supports this. However, the multitude of possibilities shows that a lot is moving in this context and that the way there is becoming increasingly easier.
