Wednesday, 23 July 2014

Example of a Continuous delivery pipeline

Continuous delivery has become a juicy problem again with the steer towards Virtual Machines as an Artefact and Operating System containers as an artefact. To put it short, your Vagrant managed VirtualBox/Docker.io style applications. I would highly recommend to read about Vagrant with Virtual Box and Vagrant with Docker.io before we continue.

Firstly I am assuming you have a series of Environments, for example Development, Staging and Production. I believe in the concept of Genderless applications where environment specific configuration is provisioned at deployment, you will see why this will become important!




1.       Build
a.       Compile code
b.       Run unit tests
c.        Run component tests / Small integration tests
d.       Build WAR archive (Or similar)
2.       Package
a.       Destroy Virtual Machine
b.       Copy WAR from Build stage to working directory
c.        Initialize new Virtual Machine
d.       Start the Virtual Machine
e.       Provision binary dependencies (e.g. JDK 1.7, Jetty 9)
f.        Deploy WAR to Virtual Machine
g.       Package Virtual Machine
3.       Deploy – DEV
a.       Copy packaged Virtual Machine from Package stage
b.       Deploy constructive SQL Changes
c.        Deploy to inactive Pool (For N number of Boxes)
                                                               i.      Deploy packaged Virtual Machine
                                                             ii.      Provision with DEV Environment configuration 
                                                           iii.      Start application
                                                            iv.      Smoke test/Query Health check URL of Application
d.       Smoke test pool
e.       Route traffic to new pool
f.        Deploy destructive SQL Changes
4.       System Test – DEV
a.       Run Acceptance tests against deployed pool
5.       Publish
a.       Copy packaged Virtual Machine from Package stage or Deploy – DEV Stage
b.       Publish packaged Virtual Machine to artefact repository
6.       Deploy – Staging
a.       Download packaged Virtual Machine from repository
b.       Deploy constructive SQL Changes
c.        Deploy to inactive Pool (For N number of Boxes)
                                                               i.      Deploy packaged Virtual Machine
                                                             ii.      Provision with Staging Environment configuration
                                                           iii.      Start application
                                                            iv.      Smoke test/Query Health check URL of Application
d.       Smoke test pool
e.       Route traffic to new pool
f.        Deploy destructive SQL Changes
7.       Deploy – Production
a.       Download packaged Virtual Machine from repository
b.       Deploy constructive SQL Changes
c.        Deploy to inactive Pool (For N number of Boxes)
                                                               i.      Deploy packaged Virtual Machine
                                                             ii.      Provision with Production Environment configuration 
                                                           iii.      Start application
                                                            iv.      Smoke test/Query Health check URL of Application
d.       Smoke test pool
e.       Route traffic to new pool
f.        Deploy destructive SQL Changes

There is a lot of information to take in here, but most of it is rinse and repeat deployments! Isn’t that great!

Back to my statement of genderless applications, due to the size of Virtual Machines are considerably larger than a WAR file, you wouldn’t want to package and publish the same application per environment. This is a wasteful and slow process!

If you haven’t noticed, this process is quite messy. When we re-route traffic in the deployment stages, we have not cleaned up the old pool. I will leave this up to you when you wish to do this! Just be cautious on the speed and effectiveness of your pipeline. To me, a good continuous delivery pipeline should be reliable, simple, repeatable, and fast. Complex, unstable pipelines are a significant problem. How much time have you wasted with “Re-run it till it works”?
Another important point to make, Application deployments for each environment should be treated the same way as deploying an application to Production. Your application should never be unavailable due to deployment. Design your deployment solution to not break what is currently available. In our example we utilize Blue-Green application deployment and Constructive-Destructive SQL deployment.

Constructive-Destructive SQL deployment is a 2 phase deployment of any changes to your database. What we want to avoid is making a change to the Database which will make the currently running applications break as well as making a change to the Database so your new version will work. Examples of changes in the Constructive phase are; adding new Table, adding new Column, adding new Index. Destructive changes, as you might have guessed; deleting a Table, deleting a Column, adding a constraint.
You may be wondering why I placed the Publish stage so late on. If publishing a larger artefact takes a few minutes, it would probably be worth publishing only when you know it is a suitable release candidate (i.e. when all your automated tests are happy!). On that note, be aware of the disk spaced used in your repository. You may wish to have a purging strategy for old versions of your application.

My final parting words, this is simply an example. This solution will not fit all applications, but the principles will! Remember reliable, simple, repeatable, and fast!

2 comments:

  1. I have also found that making each step even more granular helps with developer understanding of the CI, and makes the scope of "where" the problems are smaller.

    ReplyDelete
  2. Another technology may also be interesting for you or your followers - data room software . Any type of info can be exchanged via this tool from any part of the world.

    ReplyDelete