Pipeline as Code Explained in the Simplest Possible Way

In software development, CI/CD or Continuous Integration/Continuous Delivery pipelines help in building and deploying your code to different environments through an automated process.

However, creating and maintaining this pipeline in itself might become a challenging task. Enter Pipeline as Code – an approach in which you create your entire CI/CD pipeline in code format. Rather than relying on Web User Interfaces and drag-and-drop tools, you use configuration files to define how your application code will be built, tested, and deployed.

But before going into the details of Pipeline as Code and how you can build yours, let’s first understand what a pipeline exactly is.

What is a Pipeline in Software Development?

A pipeline in software development is a series of automated steps that takes your latest code changes, runs specific processes on it, and deploys it in your environment of choice. Let’s understand this better with an example.

Imagine you have three microservices, and you’ve added new features to one of them. Now, you want to run the code-level unit tests. Once they pass, you also want to check for code formatting issues. Next, you want to build your code. After that, you want to deploy it to two different environments with multiple machines in each environment. Finally, you’d want to run integration tests to ensure your changes are in sync with the other services.

You could choose to do all the above steps manually. But that’ll take up a lot of your time, and you’d be prone to errors. So, is there a way to automate them? Yes! You can create a pipeline and define all of the steps. Thereafter, every time you make changes to your code, you can trigger the pipeline and not worry about any manual steps.

Benefits of Pipeline as Code

If you’re using tools such as drag-and-drop, then it becomes difficult for you to track changes, maintain standardization, or promote collaboration. Pipeline as Code is a better way of defining your software development pipeline.

It helps keep things consistent. By promoting automation, you get repeatability and use the same pipeline for other systems. And just like application code, the code used to define your pipeline promotes collaboration.

#1. Consistency

Being defined in textual format ensures that nothing happens out of order. By enforcing a standard workflow for all your application builds and deployments, you get consistency and reduce the risk of unexpected issues.

And with consistency, you also gain compliance checks and security. Ensuring a consistent pipeline allows you to define your security scans and vulnerability checks such that nothing gets past you.

#2. Repeatability

Create your pipeline and set automation. In addition to consistency, your automated pipeline ensures every application code goes through the same stages and checks.

Your code will flow through the same build and deployment process every time you run your pipeline. You maintain repeatability across all your runs.

#3. Collaboration

With code being the medium using which you’re creating your pipeline, you promote collaboration. Multiple people across your teams can contribute to the same code, just like you do for application code.

Pipeline as Code also allows you to maintain version control and allow code reviews. This ensures that the best practices are followed, and potential issues are caught early.

Now, let’s dive into how you can create your own pipeline using Pipeline as Code.

Pipeline as Code in Jenkins

When it comes to Continuous Integration and Continuous Deployment (CI/CD) systems, Jenkins is the leading open-source automation server. With Jenkins, you can effortlessly integrate your code changes, automate testing and building, and deploy software. And you can do all of this reliably and efficiently.

Whether you’re a hobbyist trying to learn more about automation pipelines or you’re building complex enterprise systems, Jenkins meets all of your project’s unique requirements. Its abundance of plugins and ever-growing community can help you make the most of your automation.

In Jenkins, a Pipeline is a set of various different plugins defined in a particular order that creates your CI/CD system. Be it simple or complex use cases, you can create your pipeline using code with the Pipeline domain-specific language (DSL) syntax. The DSL is built on top of Apache Groovy.

The foundation of Pipeline as Code in Jenkins is the Jenkinsfile – a text file that contains code describing all the different stages and actions. Let’s learn how to create your Pipeline as Code using the Jenkinsfile.

How to create your Pipeline as Code?

After you’ve installed and launched Jenkins, navigate to the Web UI on your browser. You might have to log in. Next, you’ll end up on the main Dashboard page. You’re going to start from here and create your pipeline.

1. On the left panel, you’ll find a New Item button.
2. Click on it to navigate to the next page.
3. Once you’re on the new page, you’ll find the prompt to create an Item.
4. Give a name in the Enter an item name field. This is mandatory.
5. Keep in mind that a directory will be created with the same name. Hence, it’s best to avoid whitespace as it may lead to unwanted side effects.
6. After that, select the Pipeline option and click on the OK button present at the bottom of the screen.
7. The Configuration window is presented.
8. Click on the Pipeline option in the left panel or scroll down to the Pipeline section.

Let’s start with a simple pipeline that you can configure directly from the UI.

Create Pipeline as Code Directly in Jenkins

Once you’re in the Pipeline section, you’re ready to create your first Pipeline as Code.

From the Definition drop-down menu, select the Pipeline script option. Below it, you’ll find a Script area where you can code your pipeline. Jenkins maintains the script created here.

Jenkins allows you to choose between two coding styles or syntaxes – Declarative Syntax and Scripted Syntax. While Declarative is easy to use and ideal for simple pipelines, the Scripted Syntax is for power users and designing complex flows.

Using Declarative Syntax, create 3 simple stages – Build Code, Test Code, and Deploy Code by using the following code snippet:

pipeline {
    agent any

    stages {
        stage('Build Code') {
            steps {
                echo 'This is the step for build...'
            }
        }
        stage('Test Code') {
            steps {
                echo 'This is the step to test...'
            }
        }
        stage('Deploy Code') {
            steps {
                echo 'This step deploys the code...'
            }
        }
    }
}

You can also use the Scripted Syntax as shown below:

node {
    stage('Build Code') {
        echo 'This is the step for build...'
    }
    stage('Test Code') {
        echo 'This is the step to test...'
    }
    stage('Deploy Code') {
        echo 'This step deploys the code...'
    }
}

Click on Save. Now, Click on the Build Now button present in the left panel. This will trigger the pipeline you’ve just created.

Once your pipeline finishes, you can check it in the Build History. If this is your first run, click on the build number #1 present. Next, click on Console Output present in the left panel. You’ll find the 3 echo statements that you have in your pipeline code at each stage.

Create Pipeline as Code using an External File

It becomes challenging to maintain your pipeline directly in Jenkins when it starts to become complex. In this case, you’d want to create an external file and use it.

Before you create your Jenkins pipeline, you need an external repository and a version control system. Let’s create a Git repository and host it remotely on GitHub. You’ll create your Jenkinsfile and store it here.

1. Go to your GitHub profile. You can create a free account if you don’t have one.
2. Create a new repository. Name it customJenkins.
3. On your local machine, ensure that you have Git installed.
4. Create a directory in your location of choice.
5. Navigate inside the directory and open your terminal.
6. Initialize an empty Git repository using the git init command.
7. Now, create a new file, which will be your Jenkinsfile. Let’s name it customJenkinsfile.
8. Write your Pipeline as Code inside this file. As an example, use the one mentioned below:

pipeline {
    agent any

    stages {
        stage('Build Code') {
            steps {
                echo 'This is the step for build defined in custom file...'
            }
        }
        stage('Test Code') {
            steps {
                echo 'This is the step to test defined in custom file...'
            }
        }
        stage('Deploy Code') {
            steps {
                echo 'This step defined in custom file deploys the code...'
            }
        }
    }
}

9. Add the newly created file to Git using the command git add --all in your terminal.
10. Commit the file to Git using the command git commit -m "Created a custom jenkinsfile".
11. Link your local Git repository to your remote repository using git remote add origin [email protected]:/customJenkins.git.
12. Next, upload the file to remote (GitHub) using git push --set-upstream origin master.

You’ve now created a remote repository on GitHub that contains a custom Jenkinsfile. Let’s configure Jenkins to use this.

Configure Jenkins to use Jenkinsfile from GitHub

1. Open your Jenkins dashboard.
2. Create a new pipeline, or click on Configure in the left panel from an existing pipeline.
3. Scroll down to the Pipeline section.
4. From the Definition drop-down, select the Pipeline script from SCM option.
5. Select Git in the SCM option.
6. Provide your GitHub repository link in the Repository URL under Repositories.
7. Ensure the Branch Specifier is set as */master under Branches to build.
8. Scroll down to Script Path. Here, provide the name of the Jenkins file as customJenkinsfile. Click on Save.

After that, run your pipeline. Jenkins will first fetch your code from the remote repository. It will then create a pipeline using the custom Jenkinsfile and run all the stages.

You’ve now successfully created your own software development pipeline using Pipeline as Code. Additionally, you’ve enabled version control on your pipeline script. Any changes you make to your pipeline code can now be tracked with each Git commit. Next, it’s time to look at the best practices.

Best Practices for Writing Effective Pipeline as Code

Let’s look at the best practices you should follow when writing your Pipeline as code.

• Keep your pipeline clean and avoid writing too many complex conditions.
• If you’re doing too many commands inside the pipeline, then break them into different steps.
• Use external files with version controlling for your Pipeline as Code scripts.
• Utilize the features of the coding language, like Groovy, to integrate different steps.
• Avoid calls to Jenkins.getInstance or its accessors to mitigate security and performance issues.
• Do not overwrite built-in pipeline commands like sh and timeout.
• Create external tools or scripts for complex CPU-intensive tasks and plug them into your pipeline.
• Leverage the wide variety of existing plugins available for Jenkins to tackle your use case.
• Ensure you’ve incorporated exception and error handling, as things can go wrong.
• Do not make your Pipeline as Code tightly coupled with a lot of business logic inside it.
• Use parameterized arguments wherever possible to make your pipeline reusable.

Pipeline as Code: A Simple Approach for Complex Processes

In summary, Pipeline as Code simplifies the automation of your CI/CD pipeline by representing the entire process as code. While CI/CD pipelines automate the building, testing, and deployment of your code changes, leveraging Pipeline as Code takes it a step further. It allows you to define your workflow in text rather than rely on graphical interfaces.

With Pipeline as Code, you ensure that each step in your workflow occurs in the correct order. You reduce the risk of running into unwanted issues. Additionally, you get several benefits -including consistency, repeatability, and promoting collaboration.

Using this guide, you now know how to create your own pipelines using Jenkins, a widely used CI/CD tool. Jenkins offers a powerful and flexible platform for implementing Pipeline as Code through its Jenkinsfile. Follow the best practices and create workflows that tackle all your pipeline use cases.

If you’re looking to learn more about Jenkins, you can check out how to create your own Jenkins pipeline.