DevOps (1)


Essential Jenkins CI

The great enablers of any successful DevOps is Continuous Integration (CI), Delivery and Deployment(CD). CI is a workflow at the core while delivery and deployment are extensions on top of that workflow. Described plainly it is the automation of manual work within software development. I have written about this before.

But in essence it is about letting something neutral build and compile all new and existing code, run unit and integration tests all the time as new changes are pushed. This can, and should, of course be run by the same developer who should also create more F.I.R.S.T class tests. It would be wrong to say that CI is about distrusting developers to be professionals. But it is about creating safeguards from accidents. By failing the build when something is wrong as early as possible, and also notifying stakeholders and developers that committed since the last build, we have a better change to fix it early.

This also has the side-effect that it encourages a workflow that makes developers to rely on a clean, compiling mainline with green tests. sometimes this also leads to a state where the mainline is always releasable or better yet, always deployed to production. It becomes a point of truth to the current state of the code. This also encourages creating more, and better, tests at all levels in the test-pyramid.

The way most teams accomplish this is by using a build or CI server. Now these days there are a few on the market, but the most popular one by far is Jenkins, likely because it is the only one licensed under the Creative Commons licensing, giving a’lot of flexibility to a very vibrant plugin community which likely contributes to its ever growing popularity. That is why this article will describe the essentials when working with Jenkins.

Meet Jenkins

At its core, Jenkins is not much more than a bunch of cron jobs with a web frontend. It is highly extendable with plugins that affect the system as a whole or that adds additional build-steps and triggers to a job.

Jobs

You can make a job run practically any build system for any language and collect the reports and results to the job dashboard. It can be integrated to other platforms such as Sonar to statically analyze code and look for obvious bugs. Jobs can also be made a part of the code-review process even before code is merged to a mainline. The flexibility is what makes it powerful.

In the dashboard you can also find the workspace link that takes you to the folder where the project has been built. This is good if you are debugging or setting your job up to understand what is happening behind the console log as you can inspect the files here.

General

Every job consists of a general section that allows you to describe, provide links to source control, and define any parameters that must be provided when triggering (starting) the job. Parametarized tests are a great way to provide a simple interface for less-technical users in Jenkins. Like running system tests towards different environments using a subset of all testcases. Where a choice parameter becomes a dropdown of different test servers to deploy it to and a CSV String parameter specifies the testcases to run.

SCM

This is the part where you tell Jenkins where to checkout code from. By default it supports GIT and SVN for source control. It can also support other SCMs through plugins such as Mercurial and CVS, why you would ever need to use them when GIT exists, but they exist.

Build Triggers

Jobs can be configured to trigger at specific times, after other jobs have executed, manually or trigger on SCM. SCM is a trigger on something new that has been pushed to a source code repository. But there can also be triggers that can come webhooks from pull-requests (GitHub, BitBucket), merge-requests (GitLab), Patch-sets(Gerrit) and so on.

Environment

Some plugins can inject variables that can be used throughout the job, use secrets and managed property files. These variables can usually be used in all fields that a configurable in the Jenkins job using the $VARIABLE notation. Jenkins also has a bunch of default environment variables that you can use at your leisure.

Build Steps

These steps are the bread and butter of a job, and you can do anything from write pure sh and bat scripts inline that Jenkins should execute to run build systems such as ANT, Maven, Gradle, MSBuild or dotnet to triggering SonarScanner in the OS. If you find yourself having more than five build steps then you should probably consider breaking the job up into several jobs instead of a super-job.

Post steps

These steps are used mostly to collect test and analysis reports from the workspace to show in the dashboard. It is also used to send notifications by email with the email notification step, slack or even call remote APIs to notify code-reviews that the build succeeded.

Standard Jobs in most projects

A common setup of Jenkins jobs that can usually be set up no matter which stack or systems you are building. Ranked in the order of priority.

  • CI Build: Compile, test and build artifacts that triggers on SCM against mainline, artifacts deployed to artifact storage to be possibly deployed later.
  • Sonar CI Build:: Static code analysis that triggers daily, weekly, monthly or after a release.
  • Pre-merge Build: Compile and test that triggers as part of code-review
  • Sonar pre-merge Build: Static code analysis that triggers as part of code-review and reviews the change
  • Deploy to test job: Deploy the latest artefact from CI Build or specific branch.
  • Release Job: Perform a release, deploy the release to artifact storage and deploy it to production.

Good luck with your Jenkins endeavours adopting CI/CD!

Comment on below if you found this useful or if i said something stupid.