Category Archives: Devops

Devops

How to use the variable group at runtime in Azure YAML Pipeline

Leave a comment

When & Where to use?

We received the request that we would like to pass the variable group as a runtime parameter so that whenever I run the pipeline, it should allow me to select the variable group name as input, and based on the input value for the variable group during runtime my pipeline should proceed. In this article, we will discuss How to use the variable group at runtime in Azure YAML Pipeline.

This can be achieve by using the Runtime parameters. Runtime parameters let you have more control over what values can be passed to a pipeline. In this article 

What is Runtime parameters?

You can specify parameters in templates and in the pipeline. Parameters have data types such as number and string, and they can be restricted to a subset of values. The parameters section in a YAML defines what parameters are available. These runtime parameters allow you to have more control over the parameter values you pass to your pipelines.

Parameters are only available at template parsing time. Parameters are expanded just before the pipeline runs so that values surrounded by ${{ }} are replaced with parameter values. Use variables if you need your values to be more widely available during your pipeline run.

Note: If you are going to trigger the pipeline manually then you can make use of Runtime parameters in the Azure DevOps pipeline.

Runtime parameters let you have more control over what values can be passed to a pipeline. Unlike variables, runtime parameters have data types and don’t automatically become environment variables.

Let we see How to use the variable group at runtime in Azure YAML Pipeline

Step 1: Define the parameters under the Values section

Ensure Always Set runtime parameters at the beginning of a YAML. This example pipeline accepts the value of variable and then outputs the value in the job

parameters:
- name: variable_group
displayName: Variable Group
type: string
default: app-sitecore-dev
values:
- app-sitecore-dev
- app-sitecore-qa
- app-sitecore-pprd
- app-sitecore-prd
- app-sitecore-pprd-hotfix

trigger: none # trigger is explicitly set to none

Step 2: Assign the selected value to the variable group.

Post slection of variable group during manula build, the selected variable will be assinged by using ${{ parameters.<parameter_name> }}. once runtime parameter is assinged the sequence of stage/jobs can able to use the values

variables:
- group: ${{ parameters.variable_group }}

Step 3: Use the values from the selected variable group

Based on the variable group assinged from run time parameter, the remaining stage can fetch the value from the variable group like agentPool…

stages:
- stage: Build_Artifacts
jobs:
- template: Prepare_Artifacts.yml
parameters:
agentPool: '$(agentPool)'
TargetFolder: '$(Build.ArtifactStagingDirectory)'

Full YAML Code

parameters:
- name: variable_group
  displayName: Variable Group
  type: string
  default: app-sitecore-dev
  values:
  - app-sitecore-dev
  - app-sitecore-qa
  - app-sitecore-pprd
  - app-sitecore-prd
  - app-sitecore-pprd-hotfix

trigger: none # trigger is explicitly set to none

variables:
- group: ${{ parameters.variable_group }}

stages:
- stage: Build_Artifacts
jobs:
- template: Prepare_Artifacts.yml
parameters:
agentPool: '$(agentPool)'
TargetFolder: '$(Build.ArtifactStagingDirectory)'

Output

Devops

Bash Scripting – If Statement

Leave a comment

The Bash Scripting  is now a days mandatory language for most of the system admins/devops guys. so in upcoming articles we will shed light on the power and subtlety that is the Unix shell, I’d like to take a dive into just one of its many features: Bash Scripting – If Statement.

When coding, you might need to make decisions based on certain conditions. Conditions are expressions that evaluate to a boolean expression (true or false)Statements that help to execute different code branches based on certain conditions are known as conditional statements.if…else is one of the most commonly used conditional statements. Like other programming languages, Bash scripting also supports if…else statements. And we will study that in detail in this blog post.

In another way, If statements (and, closely related, case statements) allow us to make decisions in our Bash scripts. They allow us to decide whether or not to run a piece of code based upon conditions that we may set.

SYNTAX

When you are using a single if statement, the syntax is as follows: A basic if statement effectively says, if a particular condition is true, then perform a given set of actions. If it is not true then don’t perform those actions. If follows the format below:

The if statement is composed of the if keyword, the conditional phrase, and the then keyword. The fi keyword is used at the end of the statement. The COMMANDS gets executed if the CONDITION evaluates to True. Nothing happens if CONDITION returns False; the COMMANDS are ignored.. The basic syntax of an if statement is the following:

if [ condition ]
then
    statement/actions
fi

The “[ ]” in the if statement above are actually a reference to the command test. This means that all of the operators that test allows may be used here as well. When you are using a multiple condition check with if statement, the syntax is as follows:

if [ condition ] ; then
   statement/actions
elif [ condition ] ; then
   statement/actions
else
   statement/actions
fi
  • if >> Perform a set of commands if a test is true.
  • elif >> If the previous test returned false then try this one.
  • else >> If the test is not true then perform a different set of commands.

Note that the spaces are part of the syntax and should not be removed.

Example: Simple with IF statement

Let’s go through an example where we are comparing two numbers to find if the first number is the smaller one.

a=25
b=30

if [ $a -lt $b ]
then
    echo "a value is less than b"
fi

Output: a value is less than b

Example: How to Use the if .. else Statement

Let’s see an example where we want to find if the first number is greater or smaller than the second one. Here, if [ $a -lt $b ] evaluates to false, which causes the else part of the code to run.

a=65
b=35

if [ $a -lt $b ]
then
   echo "a is less than b"
else
   echo "a is greater than b"
fi

Output: a value is greater than b

Example: How to Use if..elif..else Statements

To have comparisons, we can use AND -a and OR -o operators as well in the bash command. For performing the checks between two values, we can use AND -a and OR -o as well.

In this example, we will do the check on 3 values conditions:

if [ $a == $b -a $b == $c -a $a == $c ]
then
   echo "All values are equal"

elif [ $a == $b -o $b == $c -o $a == $c ]
then
   echo "May be more than one value is equal"

else
   echo "All numbers are not equal"

fi

Conclusion on Bash Scripting – If Statement

You can check the inputs based on conditions like if..else and make the code more dynamic. In this tutorial, hope you learned Bash Scripting – If Statement

I hope you found this tutorial helpful.

What’s your favorite thing you learned from this tutorial? Let me know on Twitter!

Devops

Using secrets from Azure Key Vault in a pipeline

Leave a comment

You know as a best practice, DevOps guys need to ensure all the secrets need to be kept inside the Keyvalut instead of using directly from the Azure DevOps Variable group. So, in this article, we are going to see how we can do Variable substitute from KeyVault in YAML Azure DevOps pipelines (ie., Using secrets from Azure Key Vault in a pipeline) 

Config File

Below is the sample config file which we are going to use for substituting variables from Key Vault in YAML Azure DevOps pipelines

Step 1: Fetch the Key from Key vault:

The variable substitution can be done with 2 tasks in Azure DevOps, let’s start. The task can be used to fetch the latest values of all or a subset of secrets from the vault and set them as variables that can be used in subsequent tasks of a pipeline. The task is Node-based and works with agents on Linux, macOS, and Windows. First, we need to create the task for Connecting and fetching the secrets from the Azure Keyvalut. As we mentioned RunAsPreJob: false so the value will only scope up to the next following task alone.

- task: AzureKeyVault@2
  inputs:
    azureSubscription: 2a28a5af-3671-48fd-5ce1-4c144540aae2
    KeyVaultName: kv-dgtl-dev
    SecretsFilter: 'smtp-host,smtp-username,smtp-password'
    RunAsPreJob: false

Point to remember for Variable substitute from KeyVault:

  • RunAsPreJob – Make secrets available to the whole job, Default value is false
  • Keyvalut task needs to run before the job execution begins. Exposes secrets to all tasks in the job, not just tasks that follow this one.
  • Ensure the Agent machine has the required permissions to access the Azure key vault
  • if you want to fetch the all secrets during this task then you can specify ‘*’ instead of secrets name in the SecretsFilter.

Step 2: Apply the secrets to config files:

Second, we can have the replace token task to have the target files which need to replace the variables. once this is executed, the value fetched from the key vault will apply to the matched variable

- task: replacetokens@5
  inputs:
    rootDirectory: 'src/Feature/Forms/code/App_Config/Include/Feature/'
    targetFiles: 'dotnethelpers.Feature.Forms.SMTP.config,SMTP_external.config'
    encoding: 'auto'
    tokenPattern: 'default'
    writeBOM: true
    actionOnMissing: 'warn'
    keepToken: false
    actionOnNoFiles: 'continue'
    enableTransforms: false
    enableRecursion: false
    useLegacyPattern: false
    enableTelemetry: true

Point to remember:

  • The token pattern is set to default (so I used #{YOUR_VARIABLE}#, it may define based on your requirement.
  • The name of the Keyvalut secrets needs to match with the config variable which needs to substitute. For example, in the config, we have variables like smtp-host, smtp-username, and smtp-password so the Azure key vault secrets name need to match with same.
Devops

Trigger Azure DevOps pipeline automatically using PowerShell

Leave a comment

In many situations, we need to trigger pipelines automatically or from another pipeline (it may be another build pipeline or release pipeline). In my project, I had the same situation where I need to trigger the build from the release pipeline, in my case, the build (CI) pipeline is written in the YAML, and the release (CD) pipeline is configured in the classic editor.

How we can trigger pipelines automatically?

Trigger pipelines automatically can be achieved using Azure tasks or using PowerShell (can be done through the API using PowerShell). Using this, you can trigger a build or release pipeline from another pipeline within the same project or organization but also in another project or organization.

In this example, we will be going to discuss how we can achieve this through PowerShell using the API. in a future post, we can discuss how we can achieve using the task in Powershell.

Step: 1 Create the PAT token for Authorization

To get started, a Personal Access Token is required with the appropriate rights to execute pipelines. To generate a new Personal Access Token follow the link:

Step: 2 Enycrpt the PAT token

Always encrypt the pat token before using it in our script and kept the pat in Keyvalut. For this example, I used direct here for our example.

$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(“:$($token)”))

Step: 3 Define the API and assign it to variable

This was the latest API version 7.0 which I am going to use for Triggering the pipeline automatically using PowerShell Azure DevOps. As the name implies, we can able to get the {organization}/{project} name easily. if you are new to Azure DevOps, they will struggle to find the {pipelineId}. please find the below snapshot for reference, after clicking on the pipeline which you need to trigger, there you are able to find the build?definitionid which is called as pipelineId.

Syntax : https://dev.azure.com/{organization}/{project}/_apis/pipelines/{pipelineId}/runs?api-version=7.0

$url=”https://dev.azure.com/myOrganization/Myproject/_apis/pipelines/4/runs?api-version=7.0″

 

step: 4 Pass the parameter in the body of API.

This action in required as there are a lot of branches in my repo and the build needs to understand from which branch the build needs to be triggered so I am going to pass the branch name for the pipeline.

$JSON = @’
{
“self”: { “refName”:”develop”},
}
‘@

Step: 5 Invoke the API to trigger pipelines automatically

In this example, I am going to use the PowerShell task to execute the below script as shown in the below snapshot to Trigger the pipeline automatically.

 

$response = Invoke-RestMethod -Uri $url -Headers @{Authorization = “Basic $token”} -Method Post -Body $JSON -ContentType application/json

Full Code

$token = '5dfdferedaztxopaqwxkzf7kk4xgfhn5x5akuvgn3tycwsehlfznq'
$url="https://dev.azure.com/myOrganization/Myproject/_apis/pipelines/4/runs?api-version=7.0"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))

$JSON = @'
{
"self": { "refName":"develop"},
}
'@

$response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "Basic $token"} -Method Post -Body $JSON -ContentType application/json

 

Devops

How to Create Your First simple Jenkins Pipeline

Leave a comment

In our first Jenkins tutorial for beginners, we focused on how to install and configure Jenkins. In this tutorial, How To Create Your First simple Jenkins Pipeline. We’ll keep it simple and avoid Maven or Git at this juncture. We’ll just create a Jenkins freestyle job that invokes the JDK’s runtime instance and prints out the version of the JRE that is currently running in the Jenkins machine (location of Jenkins installation).

STEP: 1 Login into Jenkins and go to Jenkins dashboard

To create a Jenkins freestyle job, log on to your Jenkins dashboard by visiting your Jenkins installation path. Usually, it will be hosted on localhost at http://localhost:8080 If you have installed Jenkins in another path, use the appropriate URL to access your dashboard as shown in the below Jenkins job creation example.

STEP: 2 Create New Item (New job/pipeline)

The first step to creating a Jenkins build job is to click the New Item link in the top left-hand corner of the admin console and enter the Item name & click Ok.

Note: If you are unable to see this icon, it means that you don’t have sufficient privileges. In the next window, type the name of the job such as the first job, select job type as freestyle job, and then click ok:

STEP 3: Configure the new job details

After clicking OK ( in STEP 3), the configuration page for the freestyle Jenkins job will appear as shown in the below snapshot. Notice there are a number of options to configure, including build triggers, source code management options, Jenkins build job steps, and post-build actions. As we mentioned in starting of the post, we are only going to create simple jobs without doing any build or deployment.

In the Jenkins build job, we will change the description to “My_First_JenkinsJob” and Under the “Source Code Management” section, for this example, we are not going to use any GIT URL to download the solution so you can select “None”

In this job/pipeline, we going the check the java version that was installed in our local/Jenkins machine. For this, in the “Build section” choose “Execute Windows batch command” from the drop-down and type the “java -version” in the window batch command section.

STEP 4: Save the job and click on the Build Now link.

Now your Jenkins Job is ready for checking the version of Java installed on the Jenkins machine.

STEP 5: Save the job, click on the Build icon & Check the status.

To run the created job, click on the job which you need to build. Once the new job is opened as shown in the below snapshot, click the “Build Now” to start the job execution. You can check the build status under the “Build History section” at the left bottom of the screen.

STEP 6: View the log for checking the output

In the window batch command, we placed the cmd to check the version of java.  So the same has been executed and you can able to see the output in the log file as shown in the below image.

Based on this simple example, I now hope you can able to Create Your First simple Jenkins Pipeline.

Devops

Install Jenkins on Windows – A Step-By-Step Guide

3 Comments

In this article, we will go through the steps to download and install Jenkins on Windows. Jenkins is a free and open-source automation software used for building, testing, and deploying code to achieve the end goal of Continuous Delivery and Continuous Integration. It provides faster and more efficient code deployment in multiple environments. Jenkins supports a wide range of plugins due to which it can deploy almost any kind of code to any environment.

What is Jenkins?

Jenkins is a self-contained, open source (DevOps tool) automation server which can be used to automate all sorts of tasks related to building, testing and delivering or deploying software. Jenkins can be installed through native system packages, Docker, or even run standalone by any machine with a Java Runtime Environment (JRE) installed.

Jenkins may be installed on either Windows/Unix/supported platforms, but we will focus only install Jenkins on a Windows machine in the article (the below steps explain the installation in the standalone machine).

STEP 1: Prerequisites

Before you proceed to install Jenkins in your windows/Unix system, there are some prerequisites for Jenkins to install Jenkins on your computer.

Hardware requirements:

  • Hardware requirements (Minimum): 256 MB of RAM, 1 GB of drive space (although 10 GB is a recommended minimum if running Jenkins as a Docker container)
  • Hardware configuration for a small team (Recommended) : 4 GB+ of RAM & 50 GB+ of drive space

Software Requirements:

  • You should have the latest Java software installed as a prerequisite. Since Jenkins runs on Java, you need either the latest version of Java Development Kit (JDK) or Java Runtime Environment (JRE).
  • You should have access to install Software on Windows Server.

You can refer to the prerequisites of Jenkins in Jenkins.io

STEP 2: Choose the type of Jenkins download 

Jenkins releases two types of versions based on the organization’s needs. The first one is the Long-term support release & Weekly release. First, you need to download the latest Jenkins software from Download Page. At the time of writing this article, Jenkins 2.289.3 is the latest version. This might be different for you.

Long-term support releases are available every 12 weeks. They are stable and are widely tested. This release is intended for end users.

Weekly releases are made available every week by fixing bugs in its earlier version. These releases are intended towards plugin developers.

For this article, we will use the LTS, and more of the steps will remain the same for the Weekly release.

STEP 3: Download the Jenkins tool

First you need to download the latest Jenkins software from Download Page. At the time of writing this article, Jenkins 2.346.2 is the latest version (May you will find the different version during your time).

STEP 4: Double-click on Downloaded setup

Go to download location from the local computer (unzip) and Double-click on jenkins.msi. You can also Jenkin using a WAR (Web application archive) but that is not recommended.

STEP 5: In the Jenkin Setup screen, click Next.

STEP 6: Choose the Installation location

Choose the location where you want to have the Jenkins instance installed (default location is C:\Program Files (x86)\Jenkins), then click on the Next button.

STEP 8: Service Logon

You need to provide user account credentials to run Jenkins as Independent Windows Service. These can be done using two different ways – run service as LocalSystem or run service as local or domain user. Usually, it is recommended to use a local or domain user to run the Jenkins Service but here we will run the service as LocalSystem. Then Click on Next.

STEP 9: Choose your port

By default Port 8080 will be used for running Jenkins Service but you can always change this port as per your need. Once port detail is given you can quickly check its availability by clicking on Test Port. If testing goes successful then Click on Next.

SETP 10: Select Java Home Directory

Since Jenkins requires Java Runtime Environment to run so here you need to provide the JRE path to proceed with the installation. 

STEP 11: Choose a Custom setup

If you want to install any other features during the installation process then you can select them from here and click on Next. You can also select and install other features post-Jenkins Installation.

STEP 12: Start the installation

If you see the below window then it means Jenkins is finally ready to install. You can now just click on Install to begin the installation process.

Post-installation you can verify the Jenkins service is running in the services. If you have any separate service account then you can choose “Run service as local or domain user” option during STEP 8 and provide the service account username &password instead of “run service as LocalSystem”.

Step 13: Unlock Jenkins

Type http://localhost:portno (here I configured with 8080 during the installation) in the browser. To ensure Jenkins is started securely by an administrator account, a password is written on C:\Windows\system32\config\systemprofile\AppData\Local\Jenkins\.jenkins\secrets\initialAdminPassword file needs to be given in the below screen to Continue.

Step 14: Choose the Customize Jenkins

You can choose either “Install suggested plugin” or “Select Plugins to install ” ( this option will not install any plugin and you can install based on your organization’s needs). For my setup, I have chosen the first option.

Step 15: Create an Admin User

You can also create an admin user account in case you don’t want to proceed as an admin. Here we are creating a user cyber hub and going to use the same for accessing Jenkins Server. In case you don’t want to create any user, you can click on Skip and continue as admin.

Step 16: Instance Configuration

Then you need to set up the Jenkins URL below Window. Here we will use the below default URL. So we leave it as it is and then click on Save and Finish.

Step 17: Using Jenkins in the browser

You will see a Jenkins Dashboard like below logged in with cyberithub account and now you can start creating your Job.