All posts by Thiyagu

Devops

Search and Replace String Using the sed Command in Linux/Unix.

Leave a comment

Introduction:

The sed command, a powerful stream editor in Linux/Unix, is a cornerstone for text manipulation. This guide will delve into the intricacies of using sed to search and replace strings within files. We’ll explore various scenarios, from replacing single occurrences to global substitutions, and even handling case-insensitive replacements. Whether you’re a seasoned system administrator or a budding developer, this comprehensive tutorial will equip you with the knowledge to effectively wield the sed command for your text processing needs. We will discuss more about how to Search and Replace String Using the sed.

My Requirement & solution:

We are maintaining the application in Linux machine (in AKS pods) and as a Devops team, we Got a requirement to replace some config values based on the environment (value need to be maintain in the AKS environment variable). To manage this, we thought to create one startup script in the docker image which will execute during the new image deployment ,where we used the sed command to achieve the find & replace of config value based on environments. Based on my experience i though to write this article (Search and Replace String Using the sed Command in Linux/Unix) immediately which will be helpful like me who are new to the Linux Operating system/Bash commands. 

What Is the Sed Command in Linux?

The SED command in Linux stands for Stream Editor and it helps in operations like selecting the text, substituting text, modifying an original file, adding lines to text, or deleting lines from the text. Though most common use of SED command in UNIX is for substitution or for find and replace.

By using SED you can edit files even without opening them, which is much quicker way to find and replace something in file, than first opening that file in VI Editor and then changing it.

[su_highlight color=”#2F1C6A”]Syntax: sed OPTIONS… [SCRIPT] [INPUTFILE…][/su_highlight]

  • Options control the output of the Linux command.
  • Script contains a list of Linux commands to run.
  • File name (with extension) represents the file on which you’re using the sed command.

[su_quote]Note: We can run a sed command without any option. We can also run it without a filename, in which case, the script works on the std input data.[/su_quote]

Key Advantages of Using sed

  • Efficiency: sed allows for in-place editing, eliminating the need to manually open and modify files in a text editor.
  • Flexibility: It supports a wide array of editing commands, enabling complex text manipulations.
  • Automation: sed can be easily integrated into scripts for automated text processing tasks.

Search and Replace String Using the sed

Replace First Matched String

The below example, the script will replace the first found instance of the word test1 with test2 in every line of a file

    sed -i 's/test1/test2/' opt/example.txt

The command replaces the first instance of test1 with test2 in every line, including substrings. The match is exact, ignoring capitalization variations. -i tells the sed command to write the results to a file instead of standard output.

Search & Global Replacement (all the matches)

To replace every string match in a file, add the g flag to the script (To replace all occurrences of a pattern within each line). For example

    sed -i 's/test1/test2/g' opt/example.txt

The command globally replaces every instance of test1 with test2 in the /example.txt.

The command consists of the following:

  • -i tells the sed command to write the results to a file instead of standard output.
  • s indicates the substitute command.
  • / is the most common delimiter character. The command also accepts other characters as delimiters, which is useful when the string contains forward slashes.
  • g is the global replacement flag, which replaces all occurrences of a string instead of just the first.
    “input file” is the file where the search and replace happens. The single quotes help avoid meta-character expansion in the shell.

Search and Replace All Cases

To find and replace all instances of a word and ignore capitalization, use the I parameter:

#I: The case-insensitive flag.    
sed -i 's/test1/tes2/gI' opt/example.txt

The command replaces all instances of the word test1 with test2, ignoring capitalization.

Conclusion 

The sed command is an invaluable tool for text manipulation in Linux/Unix environments. By mastering its basic usage and exploring its advanced features, you can streamline your text processing tasks and significantly improve your system administration and development workflows. This tutorial has provided a foundational understanding of sed’s search and replace capabilities. For further exploration, consider delving into more advanced sed scripting techniques and exploring its other powerful features.

I hope you found this tutorial helpful. What’s your favorite thing you learned from this tutorial? Let me know on comments!

kubernetes

How To Connect Azure Kubernetes Cluster Using Kubectl

Leave a comment

Introduction

Azure Kubernetes Service (AKS) simplifies the deployment, management, and scaling of containerized applications using Kubernetes. To interact with an AKS cluster, you need to establish a connection using kubectl, the Kubernetes command-line tool. This guide provides a step-by-step process to explain How To Connect Azure Kubernetes Cluster Using Kubectl

Why Kubectl needed?

Connecting to an AKS cluster is an essential step for managing workloads, monitoring performance, and deploying applications. This process is especially critical for:

  • Monitoring Cluster Health: Using kubectl commands to retrieve performance metrics and check node status.
  • Application Deployment: Deploying and managing containerized applications in the AKS cluster.
  • Cluster Administration: Performing administrative tasks like scaling, updating, or debugging resources within the cluster.

Whether you’re a developer or administrator, establishing this connection ensures you can effectively manage your Kubernetes environment.

To connect to an Azure Kubernetes Service (AKS) cluster using kubectl, you will need to perform the following steps:

Prerequisites ( install both Azure CLI and Kubectl)

STEP: 1 Install the Azure CLI

If you haven’t installed the Azure CLI on your local machine, you can download and install it from the official Microsoft documentation here.

STEP: 2 Install Kubectl

Install the Kubernetes command-line (click here) tool “Kubectl” , if you haven’t installed it already.

Steps to connect Azure account:

STEP: 3 Authenticate with Azure

Open command prompt, run az login command to authenticate your CLI with your Azure account. Once you run this command, you will be prompted to enter your Azure account credentials.

az login

Steps to connect Azure AKS Cluster:

Go to Azure Portal -> Kubernetes Services -> Select the required Cluster -> Overview -> Connect -> to find the entire command for the specific cluster itself or follow the below commands one by one by replacing with subscription Id, cluster name and resource group name.

STEP: 4 Set the subscription

To set subscription, run 

az account set --subscriptiond 95fe7-8d2c-4297-ad8b-a8eb08322955

STEP: 5 Generate kubeconfig file

Open command prompt, run the below command and run az aks get-credentials command to connect to your AKS cluster. The get-credentials command downloads credentials and configures the Kubernetes CLI to use them.

#Syntax: 
az aks get-credentials --resource-group <resource-group-name> --name <cluster-name>

Replace <resource-group-name> with the name of the resource group that contains your AKS cluster, and <cluster-name> with the name of your AKS cluster.

az aks get-credentials --resource-group rg-dgtl-pprd-we-01 --name aks-dgtl-pprd-we-01

Above command will create the kubeconfig file in the user root directory. To get kubeconfig file in the specific location

Connect Azure Kubernetes Cluster Using Kubectl

STEP: 6 Verify the connection

To verify that Kubectl is connected to your AKS cluster, run the Kubectl get nodes command. This command should display a list of the nodes in your AKS cluster. If the command returns the list of nodes, then you have successfully connected to your AKS cluster using Kubectl.

kubectl get nodes

Points to Remember

  • Go to Azure Portal -> Kubernetes Services -> Select the required Cluster -> Overview -> Connect -> to find the entire command for the specific cluster itself or follow the below commands one by one by replacing with subscription Id, cluster name and resource group name.
  • First we need to login to the Azure account by configuring Azure subscription and login details. And then we need to connect to Kubernetes Cluster only then we can able to run Kubectl commands.

Conclusion

Connecting to an AKS cluster using kubectl is a fundamental skill for managing Kubernetes workloads in Azure. By following this guide, you can authenticate, configure, and verify your connection to the AKS cluster seamlessly. This enables you to monitor cluster performance, deploy applications, and manage resources effectively.

As Kubernetes continues to be a vital platform for container orchestration, mastering tools like kubectl and Azure CLI is essential for efficient cluster management.

Devops

How To Copy Secrets From KeyVault To Another KeyVault In Azure

1 Comment

Introduction

Azure Key Vault is a secure cloud service for managing secrets, encryption keys, and certificates. In modern multi-region deployments, ensuring that application secrets are consistently available across regions is essential for high availability and disaster recovery. However, manually copying secrets from one Key Vault to another can be tedious, error-prone, and time-consuming, especially when dealing with numerous secrets.

This blog post demonstrates how to automate the process of copying secrets from one Azure Key Vault to another using a PowerShell script. By following this guide, you can efficiently replicate secrets between regions, ensuring consistency and reducing manual intervention.

Use Case:

In our application setup, we aimed to configure high availability by deploying the application in two Azure regions. The primary Key Vault in region 1 contained numerous secrets, which we needed to replicate to the Key Vault in region 2. Manually moving each secret one by one was impractical and error-prone.

To overcome this, we developed an automated process using PowerShell to copy all secrets from the source Key Vault to the destination Key Vault. This approach eliminates human errors, saves time, and ensures seamless secret replication for high availability.

e. This blog will help you to understand How To Copy Secrets From KeyVault To Another In Azure using PowerShell script.

To clone a secret between key vaults, we need to perform two steps:

  1. Retrieve/export the secret value from the source key vault.
  2. Import this value into the destination key vault.

You can also refer below link to learn how to maintain your secrets in key vault and access in YAML pipeline

Using secrets from Azure Key Vault in a pipeline

Step 1: Install Azure AZ module

Use the below cmdlet to Install the Azure PowerShell module if not already installed

# Install the Azure PowerShell module if not already installed
  Install-Module -Name Az -Force -AllowClobber

Step 2: Set Source and destination Key Vault name

# Pass both Source and destination Key Vault Name
Param( [Parameter(Mandatory)] 
[string]$sourceKvName, 
[Parameter(Mandatory)] 
[string]$destinationKvName )

Step 3:  Connect the Azure portal to access the Key Vault (non-interactive mode)

As we are doing the automation, so you can’t use Connect-AzAccount (which will make the popup to authenticate), if want to execute without any manual intervention then use az login with non-interactive mode as shown in below.

# Connect to Azure portal (you can also use Connect-AzAccount)
az login --service-principal -u "0ff3664821-0c94-48e0-96b5-7cd6422f46" -p "XACccAV2jXQrNks6Lr3Dac2B8z95BAt~MTCrP" --tenant "116372c23-ba4a-223b-0339-ff8ba7883c2"

Step 4:  Get the all the secrets name from the source KV

# Get all the Source Secret keys
$secretNames = (Get-AzKeyVaultSecret -VaultName $sourceKvName).Name

Step 5: Copy Secrets From source to destination KV.

The below script will loop based on the number of key names to fetch both name of the key and its value from the source key Vault and started to set the key and value in the destination KvName.

# Loop the Secret Names and copy the key/value pair to the destination key vault
$secretNames.foreach{
Set-AzKeyVaultSecret -VaultName $destinationKvName -Name $_ `
-SecretValue (Get-AzKeyVaultSecret -VaultName $sourceKvName -Name $_).SecretValue
}

Full code

# Pass both Source and destination Key Vault Name
Param(
[Parameter(Mandatory)]
[string]$sourceKvName,
[Parameter(Mandatory)]
[string]$destinationKvName
)

# Connect to Azure portal (you can also use Connect-AzAccount)
az login --service-principal -u "422f464821-0c94-48e0-96b5-7cd60ff366" -p "XACccAV2jXQrNks6Lr3Dac2B8z95BAt~MTCrP" --tenant "116372c23-ba4a-223b-0339-ff8ba7883c2"

# Get all the Source Secret keys
$secretNames = (Get-AzKeyVaultSecret -VaultName $sourceKvName).Name

# Loop the Secret Names and copy the key/value pair to the destination key vault
$secretNames.foreach{
Set-AzKeyVaultSecret -VaultName $destinationKvName -Name $_ `
-SecretValue (Get-AzKeyVaultSecret -VaultName $sourceKvName -Name $_).SecretValue
}

Conclusion

Managing secrets across multiple Azure regions can be challenging but is crucial for ensuring high availability and disaster recovery. Automating the process of copying secrets between Key Vaults not only streamlines the operation but also enhances reliability and reduces the risk of errors.

By following the steps outlined in this blog, you can easily replicate secrets between Azure Key Vaults using PowerShell. This solution ensures that your applications in different regions are configured with consistent and secure credentials, paving the way for robust and scalable deployments.

Implement this process to save time, minimize errors, and focus on scaling your applications while Azure handles secure secret management for you.

 

PowerShell

Where to use the –replace operator and Replace() method & its difference

Leave a comment

Introduction

Manipulating text is a common task in any scripting or programming language, and PowerShell is no exception. Whether you need to replace a single word, a pattern, or handle more complex transformations using regular expressions, PowerShell provides two powerful tools for the job: the .Replace() method and the -replace operator.

In this article, we’ll explore the differences between these two approaches, highlight their strengths, and demonstrate practical examples to help you choose the right one for your needs. From simple string substitutions to advanced regex-based replacements, you’ll learn how to effectively use these features to handle text in PowerShell scripts with ease.

By the end of this tutorial, you’ll understand:

  • When to use the .Replace() method for straightforward replacements.
  • How the -replace operator leverages the power of regular expressions for more complex scenarios.
  • Common pitfalls and tips for working with special characters.

Let’s dive into how you can make the most of these string replacement techniques in your PowerShell scripts!

–replace operator and Replace() method

In this post, you’re going to learn where to use the PowerShell replace() method and PowerShell replace operator. The tutorial will cover the basics and even drive into some regular expressions.

.Replace is a .NET method and -replace is a PowerShell operator that uses regular expressions. In another word, the .Replace() method comes from the .NET String class whereas the -Replace operator is implemented using System.Text.RegularExpressions.Regex.Replace().

The -replace operator takes a regular expression (regex) replacement rule as input and replaces every match with the replacement string.

When and where to use it?

Like other languages, PowerShell can work with strings and text. One of those useful features is to use PowerShell to replace characters, strings, or even text inside of files. In PowerShell, Replace() method and -replace operator is used to finding specified characters and replace them with a new string. To perform simple replacements, you can use the replace() method but if you need to match and replace anything more advanced, always use the replace operator.

.Replace Method:

Example 1: Replace characters in strings.

$string = ‘hello, dotnet-helpers.com’

In the above code, we gave the string that likes to replace. From the above string value, we would like to replace “hello” with “Welcome”. To do this in PowerShell, first you need to figure out the matched text. Once it’s found, need to replace that text with a user-defined value.

$string = ‘hello, dotnet-helpers.com’
$string.replace(‘hello’,’Welcome’)

The replace() method has two arguments; the string to find and the string to replace. As shown above, the “hello” string is going to replace with “Welcome”.

Points to Remember:

You can call the replace() method on any string to replace any literal string with another. If the string-to-be-replaced isn’t found, the replace() method returns nothing.

Example 2: Replacing multiple strings

You aware that replace() method returns a string, to replace another instance, you can append another replace() method at the end ( .replace(‘dotnet-helpers’,’dotnet-helpers.com!!!’) ). In the previous example, we try to replace “hello” with “Welcome”, in this example we trying to replace another string with one more .replace method as shown below.

$string = ‘hello, dotnet-helpers’
$string.replace(‘hello’,’welcome’).replace(‘dotnet-helpers’,’dotnet-helpers.com!!!’)

Points to Remember:

You can chain together as many replace() method calls as necessary

-Replace Operator:

The replace operator is similar to the .Replace method (in that you provide a string to find and replace). But, it has one big advantage; the ability to use regular expressions to find matching strings.

Example 1: Replacing single string

$string = ‘hello, dotnet-helpers.com’
$string -replace ‘hello,’, ‘Welcome to’

Example 2: Replacing multiple strings

Like the replace() method, you can also chain together usages of the replace operator.

$string = ‘hello, dotnet-helpers’
$string -replace ‘hello,’,’Welcome to’ -replace ‘dotnet-helpers’,’dotnet-helpers.com!!!’

-Replace Operator with Regex:

Replacing strings in PowerShell with the replace() method works but it’s limited. You are constrained to only using literal strings. You cannot use wildcards or regex. If you’re performing any kind of intermediate or advanced replacing, you should use the replace operator.

The -replace operator takes a regex (regular expression) replacement rule as input and replaces every match with the replacement string. The operator itself is used as shown in the following examples : <input string> -replace <replacement rule>,<replacement string>

Example 1: With Simple Regex

In this example, you can use the expression hello|hi to match both required strings using the regex “or” (|) character as you can see below. In the below regex, it finds the match for a string like “hello” or “hi” and if a match is found it will replace with the given string.

$string = ‘hi, dotnet-helpers.com’
$string -replace ‘hello|hi’,’Good day’

Example 2: Direct Replace of special character

As per the below example, you need to replace text in a string. That string contains a couple of regex special characters like a bracket and an exclamation mark. If you try to replace the string [dotnethelpers!] with dotnet-helpers.com as shown below, then it will not work as expected because the characters will have special meaning in regex language.

$string = “hi, [dotnethelpers!]”
$string -replace ‘[dotnethelpers!]’,’dotnet-helpers.com’

The problem is you often need to replace “[]”, “!” or other characters that have special meaning in regex language. One way to achieve this is to escape every special character by “\”.

To overcome this problem, you have two options. You can either escape these special characters by prepending a backslash to the front of each character or using the Escape() method (([regex]::Escape(‘[dotnethelpers]’)).

Points to Remember:

If you try to replace any special characters directly from the string using Replace operator then it won’t work correctly as characters will have special meaning in regex language.

Conclusion :

Replacing characters or words in a string with PowerShell is easily done using either the replace method or -replace operator. When working with special characters, like [ ], \ or $ symbols, it’s often easier to use the replace() method than the operator variant. Because this way you don’t need to escape the special character.

To perform simple replacements, you can use the replace() method but if you need to match and replace anything more advanced, always use the replace operator.

PowerShell

How to Use PowerShell for DNS Record Monitoring and Troubleshooting

1 Comment

Introduction

DNS (Domain Name System) records are the backbone of internet connectivity for web servers, mail servers, and other networked systems. Missing or misconfigured DNS records can lead to service disruptions, causing websites to become unreachable or emails to fail. PowerShell offers a robust cmdlet, Resolve-DnsName, that enables administrators to monitor, validate, and troubleshoot DNS configurations (PowerShell DNS record monitoring).

This article introduces a PowerShell script designed to simplify the process of checking DNS resolution for multiple domains. By leveraging this script, you can automate DNS checks, verify propagation of updates, and ensure compliance with best practices for security and network reliability.

The script processes a list of domain DNSLists and retrieves their resolution status, IP addresses, and any associated errors, providing a comprehensive overview in a single execution.

If you’re managing web or mail servers, you know how heavily these servers rely DNS records. Error/Missing DNS records can cause all sorts of problems, including users not being able to find your website or the non-delivery of emails. It is a good thing that the PowerShell Resolve-DnsDNSList cmdlet exists, and with it, monitoring DNS records can be automated through scripting.

This script is designed to check the DNS resolution status for a list of domain DNSLists against a set of DNS servers, and it returns detailed results including the IP address, status, and any error messages.

When to Use This Script

You might use this script in several scenarios:

Monitoring and Troubleshooting DNS Issues: If you’re managing multiple domains and want to verify that they resolve correctly across different DNS servers, this script provides a quick way to gather and assess that information.

Validation of DNS Configurations: When you’re updating DNS records (e.g., for a new website, email configuration, or any other service), you can use this script to confirm that the records have propagated properly and are resolving as expected.

Security and Compliance Checks: Ensuring that all your domains are resolving correctly can be a part of your security checks to avoid phishing or man-in-the-middle attacks due to incorrect or hijacked DNS records.

Network Operations: In a larger network or data center, you might want to regularly check the resolution of internal or external domains to ensure consistent access to critical services.

Explanation for PowerShell DNS record monitoring Script

Here’s a breakdown of how the script works:

# Its an array containing the domain DNSLists you want to check.
$DNSLists = @('dotnet-helpers.com','google.org','XXXXYYY.local')

# This is an array that will store the results of the DNS resolution for each domain.
$FinalOutput = @()

foreach ($DNSList in $DNSLists) {
    # Creates an empty object with properties DNSList, IPAddress, Status, and ErrorMessage.
    $dnsObj = "" | Select-Object DNSList, IPAddress, Status, ErrorMessage
    try {
        # Attempts to resolve the domain name and filters for 'A' records
        $dnsRecord = Resolve-DnsName $DNSList -ErrorAction Stop | Where-Object { $_.Type -eq 'A' }
        #  Assigns the current domain name to DNSList
        $dnsObj.DNSList = $DNSList
        # ($dnsRecord.IPAddress -join ','): Joins any IP addresses returned by the DNS query into a single string
        $dnsObj.IPAddress = ($dnsRecord.IPAddress -join ',')
        # 'OK': Sets the status to OK if the DNS resolution is successful
        $dnsObj.Status = 'OK'
        $dnsObj.ErrorMessage = ''
    }
    # Handles any errors that occur during the DNS resolution.
    catch {
        $dnsObj.DNSList = $DNSList
        $dnsObj.IPAddress = ''
        $dnsObj.Status = 'NOT_OK'
        $dnsObj.ErrorMessage = $_.Exception.Message
    }
    # Adds the result object to the $FinalOutput array.
    $FinalOutput += $dnsObj
}
return $FinalOutput

Conclusion

DNS record validation is a critical task for maintaining the functionality and security of web and mail servers. The PowerShell script outlined in this article provides a reliable way to monitor DNS resolution for multiple domains. By automating the DNS-checking process, administrators can quickly identify and address issues, ensuring minimal downtime and improved reliability of services.

Whether you’re validating recent DNS updates, troubleshooting network issues, or performing routine security audits, this script serves as an efficient tool for managing DNS records. Its ability to handle errors gracefully and provide detailed output makes it a valuable addition to any IT professional’s toolkit.

 

PowerShell

Cache Purging in Azure Front Door with Azure PowerShell and CLI

Leave a comment

Introduction

Azure Front Door is a global, scalable entry point for fast delivery of your applications. It provides load balancing, SSL offloading, and caching, among other features. One critical task for maintaining optimal performance and ensuring the delivery of up-to-date content is cache purging. This article provides a step-by-step guide to performing cache purging in Azure Front Door using Azure PowerShell and the Azure Command-Line Interface (CLI).

What is Cache Purging?

Cache purging, also known as cache invalidation, is the process of removing cached content from a caching layer. This is essential when the content served to the end users needs to be updated or deleted. In the context of Azure Front Door, purging ensures that the latest version of your content is delivered to users instead of outdated cached versions.

Prerequisites for Cache Purging in Azure Front Door

Step 1: Open Azure PowerShell

Open your preferred PowerShell environment (Windows PowerShell, PowerShell Core, or the PowerShell Integrated Scripting Environment (ISE)).

Step 2: Sign in to Azure

Sign in to your Azure account using the following command:

Connect-AzAccount

Step 3: Select the Subscription

If you have multiple subscriptions, select the appropriate subscription:

Select-AzSubscription -SubscriptionId "your-subscription-id"

Step 4: Cache Purge using PowerShell

Method 1: Using Invoke-AzFrontDoorPurge

Purpose: Invoke-AzFrontDoorPurge is used specifically for purging content from the Azure Front Door caching service.

Usage: This cmdlet is part of the Azure PowerShell module and is used to remove specific cached content from the Azure Front Door service (ie., Cache Purging in Azure Front Door).

Use the Invoke-AzFrontDoorPurge cmdlet to purge the cache. You’ll need the name of your Front Door profile and the list of content paths you want to purge.

Here’s an example:

# prerequisite Parameters

$frontDoorName = "your-frontdoor-name"
$resourceGroupName = "your-resource-group-name"
$contentPaths = @("/path1/*", "/path2/*")

Invoke-AzFrontDoorPurge -ResourceGroupName $resourceGroupName -FrontDoorName $frontDoorName -ContentPath $contentPaths

This command purges the specified paths in your Front Door profile.

When to Use:

When you need to remove cached content specifically from Azure Front Door using Azure PowerShell.
Ideal for scenarios involving global load balancing and dynamic site acceleration provided by Azure Front Door.

Method 2: Using Clear-AzFrontDoorCdnEndpointContent

Purpose: Clear-AzFrontDoorCdnEndpointContent is used for purging content from Azure CDN endpoints, which might also be linked to an Azure Front Door service. However, it specifically targets the CDN layer.

Usage: This cmdlet clears content from Azure CDN endpoints, which can be part of a solution using Azure Front Door.

$endpointName = "your-cdn-endpoint-name"
$resourceGroupName = "your-resource-group-name"
$contentPaths = @("/path1/*", "/path2/*")

Clear-AzFrontDoorCdnEndpointContent -ResourceGroupName $resourceGroupName -EndpointName $endpointName -ContentPath $contentPaths

When to Use:

  • When working specifically with Azure CDN endpoints.
  • Useful for content distribution network scenarios where you need to clear cached content from CDN endpoints.

Step 5: Cache Purge using Azure CLI

Method 3: Using Clear-AzFrontDoorCdnEndpointContent

Purpose: az afd endpoint purge is an Azure CLI command used for purging content from Azure Front Door endpoints.

Usage: This command is used within the Azure CLI to purge specific content paths from Azure Front Door.

frontDoorName="your-frontdoor-name"
resourceGroupName="your-resource-group-name"
contentPaths="/path1/* /path2/*"

az afd endpoint purge --resource-group $resourceGroupName --profile-name $frontDoorName --content-paths $contentPaths

When to Use:

  • When you need to purge cached content from Azure Front Door using Azure CLI.
  • Suitable for users who prefer command-line tools for automation and scripting.

Key Differences

Service Targeted:

  1. Invoke-AzFrontDoorPurge: Specifically targets Azure Front Door.
  2. Clear-AzFrontDoorCdnEndpointContent: Specifically targets Azure CDN endpoints.
  3. az afd endpoint purge: Specifically targets Azure Front Door.

Use Case:

  1. Invoke-AzFrontDoorPurge: Best for scenarios involving global load balancing and content delivery with Azure Front Door.
  2. Clear-AzFrontDoorCdnEndpointContent: Best for scenarios involving Azure CDN, which might or might not involve Azure Front Door.
  3. az afd endpoint purge: Best for users comfortable with CLI and needing to purge Azure Front Door content.

Conclusion

Understanding the differences between these commands helps you choose the right tool for your specific needs to Cache Purging in Azure Front Door. Whether you are managing caches at the CDN layer or the Azure Front Door layer, Azure provides flexible and powerful tools to help you maintain optimal performance and up-to-date content delivery.

Devops

How to Delete a Blob from an Azure Storage using PowerShell

Leave a comment

In one of my automation (Delete a Blob), I need to delete the previously stored reports (reports will always append with timestamp) on daily basis in Azure storage account in automated way in the specific container. So i need to ensure my container is available before start deleting my report. This article will have detail explain about How to Delete a Blob from an Azure Storage Account using PowerShell.

New to storage account?

One of the core services within Microsoft Azure is the Storage Account service. There are many service that utilize Storage Accounts for storing data, such as Virtual Machine Disks, Diagnostics logs (specially application log), SQL backups and others. You can also use the Azure Storage Account service to store your own data; such as blobs or binary data.

As per MSDN, Azure blob storage allows you to store large amounts of unstructured object data. You can use blob storage to gather or expose media, content, or application data to users. Because all blob data is stored within containers, you must create a storage container before you can begin to upload data.

Delete a Blob from an Azure Storage

Step: 1 Get the prerequisite inputs

As in this example i am going to delete the one the sql db (backup/imported to the storage) stored as bacpac format in the container called SQL…

## prerequisite Parameters
$resourceGroupName="rg-dgtl-strg-01"
$storageAccountName="sadgtlautomation01"
$storageContainerName="sql"
$blobName = "core_2022110824.bacpac"

Step: 2 Connect to your Azure subscription

Using the az login command with a service principal is a secure and efficient way to authenticate and connect to your Azure subscription for automation tasks and scripts. In scenarios where you need to automate Azure management tasks or run scripts in a non-interactive manner, you can authenticate using a service principal. A service principal is an identity created for your application or script to access Azure resources securely.

## Connect to your Azure subscription
az login --service-principal -u "210f8f7c-049c-e480-96b5-642d6362f464" -p "c82BQ~MTCrPr3Daz95Nks6LrWF32jXBAtXACccAV" --tenant "cf8ba223-a403-342b-ba39-c21f78831637"

Step: 3 Get the storage account to Check the container exit or not

When working with Azure Storage, you may need to verify if a container exists in a storage account or create it if it doesn’t. You can use the Get-AzStorageContainer cmdlet to check for the existence of a container.

## Get the storage account to check container exist or need to be create
$storageAccount = Get-AzStorageAccount -ResourceGroupName $resourceGroupName -Name $storageAccountName

## Get the storage account context
$context = $storageAccount.Context

Step: 4 Check the container exist before deleting the blob

We need to use Remove-AzStorageBlob cmdlet to delete a blob from the Azure storage container

## Check if the storage container exists
if(Get-AzStorageContainer -Name $storageContainerName -Context $context -ErrorAction SilentlyContinue)
{

Write-Host -ForegroundColor Green $storageContainerName ", the requested container exit,started deleting blob"

## Create a new Azure Storage container
Remove-AzStorageBlob -Container $storageContainerName -Context $context -Blob $blobName
Write-Host -ForegroundColor Green $blobName deleted

}
else
{
Write-Host -ForegroundColor Magenta $storageContainerName "the requested container does not exist"
}

Full Code:

## Delete a Blob from an Azure Storage
## Input Parameters
$resourceGroupName="rg-dgtl-strg-01"
$storageAccountName="sadgtlautomation01"
$storageContainerName="sql"
$blobName = "core_2022110824.bacpac"

## Connect to your Azure subscription
az login --service-principal -u "210f8f7c-049c-e480-96b5-642d6362f464" -p "c82BQ~MTCrPr3Daz95Nks6LrWF32jXBAtXACccAV" --tenant "cf8ba223-a403-342b-ba39-c21f78831637"

## Function to create the storage container
Function DeleteblogfromStorageContainer
{
## Get the storage account to check container exist or need to be create
$storageAccount = Get-AzStorageAccount -ResourceGroupName $resourceGroupName -Name $storageAccountName

## Get the storage account context
$context = $storageAccount.Context


## Check if the storage container exists
if(Get-AzStorageContainer -Name $storageContainerName -Context $context -ErrorAction SilentlyContinue)
{

Write-Host -ForegroundColor Green $storageContainerName ", the requested container exit,started deleting blob"
## Remove the blob in Azure Storage container
Remove-AzStorageBlob -Container $storageContainerName -Context $context -Blob $blobName

Write-Host -ForegroundColor Green $blobName deleted
}
else
{
Write-Host -ForegroundColor Magenta $storageContainerName "the requested container does not exist"
}

}
#Call the Function
DeleteblogfromStorageContainer

Output:

 

PowerShell

Exception Handling – Try Catch with Custom Error Message in PowerShell

Leave a comment

An error in a PowerShell script will prevent it from completing script execution successfully. Using error handling with try-catch blocks allows you to manage and respond to these terminating errors. In this post, we will discuss the basics of try/catch blocks and how to find or handle Custom Error Message in PowerShell.

Handling errors effectively in scripts can save a lot of troubleshooting time and provide better user experiences. In PowerShell, we have robust options to handle exceptions using try, catch, and finally blocks. Let’s dive into how you can use try-catch to gracefully handle errors and add custom error messages for better feedback.

Why Use Exception Handling in PowerShell?

Scripts can fail for many reasons: missing files, invalid input, or network issues, to name a few. With exception handling, you can capture these issues, inform users in a friendly way, and potentially recover from errors without crashing your script. Using try-catch, you can:

  • Catch specific errors.
  • Display user-friendly messages.
  • Log errors for debugging.

Syntax overview of Try/Catch

Like similar in other programming languages, the try-catch block syntax is very simple and syntax will be the same. It is framed with two sections enclosed in curly brackets (the first block is a try and the second is the catch block).

try {
# Functionality within try block
}
catch {
# Action to do with errors
}

The main purpose of using the try-catch block, we can start to manipulate the error output and make it more friendly for the user.

Example 1:

After executing the below script, the below error will be shown on the screen as output and it would occupy some space and the problem may not be immediately visible to the User. So you can use a try-catch block to manipulate the error output and make it more friendly.

without Try-Catch block

Get-content -Path “C:\dotnet-helpers\BLOG\TestFiled.txt” 

with Try Catch block

In the below script, we added the ErrorAction parameter with a value of Stop to the command. Not all errors are considered “terminating”, so sometimes we need to add this bit of code in order to properly terminate into the catch block.

try {
Get-content -Path “C:\dotnet-helpers\BLOG\TestFile.txt” -ErrorAction Stop
}
catch {
Write-Warning -Message “Can’t read the file, seem there is an issue”

}

Example 2:

Using the $Error Variable

In Example 1, we have displayed our own custom message instead of this you can display the specific error message that occurred instead of the entire red text exception block. When an error occurs in the try block, it is saved to the Automatic variable named $Error. The $Error variable contains an array of recent errors, and you can reference the most recent error in the array at index 0.

try{
Get-content -Path “C:\dotnet-helpers\BLOG\TestFiled.txt” -ErrorAction Stop
}
Catch{

Write-Warning -Message “Cant’t read the file, seem there is an issue”
Write-Warning $Error[0]

}

Example 3:

Using Exception Messages

You can also use multiple catch blocks in case if you want to handle different types of errors. For this example, we going to handle two different types of errors and planned to display different custom messages. The first CATCH is to handle if the path does not exist and the next CATCH is to handle if any error related to the driver not found.

Using try/catch blocks gives additional power in handling errors in a script and we can have different actions based on the error type. The catch block focuses on not only displaying error messages but we can have logic that will resolve the error and continue executing the rest of the script.

In this example, the file mentioned driver (G:\dotnet-helpers\BLOG\TestFiled.txt) does not exist in the execution machine, so it was caught by [System.Management.Automation.DriveNotFoundException] and executed the same CATCH block.

try{

Get-content -Path "G:\dotnet-helpers\BLOG\TestFiled.txt" -ErrorAction Stop
}
# It will execute if a specific file is not found in a specific Directory
Catch [System.IO.DirectoryNotFoundException] {

Write-Warning -Message "Can't read the file, seems there is an issue"
Write-Warning $Error[0]
}
# It will execute if the specified driver is not found in the specified path
Catch [System.Management.Automation.DriveNotFoundException]{
Write-Warning -Message "Custom Message: Specific driver is not found"
Write-Warning $Error[0]

}
#Execute for Un-Handled exception - This catch block will run if the error does not match any other catch block exception.
Catch{
Write-Warning -Message "Oops, An un-expected Error Occurred"
#It will return the exception message for the last error that occurred.
Write-host $Error[0].Exception.GetType().FullName
}

OUTPUT

Devops

How to view the secret variables in Azure DevOps

Leave a comment

Today, I will be taking about a technique using which you can view the secret variables in Azure DevOps.

Introduction

Azure DevOps supports us to store secrets within Azure DevOps variable Groups which could be used with the Pipelines. These secret variables couldn’t be viewed by us manually from the portal. Sometimes, we may want to view the password to perform some other activities.

Note: The best practice to have the secrets in Azure Key Vault and same you can read and execute in Azure pipeline in very secured way. Still some legacy projects are maintaining the secrets in Azure Variable group, so this article focus on them. You can read this to use Key Vault to handle the secrets

Using secrets from Azure Key Vault in a pipeline

What are Secrets Variables in Azure Pipelines?

Secret Variables are placeholders for values which you want to store in an encrypted format and use while using running a pipeline. Secret Variables can be used for values like username, password, API key etc. Secret variables are encrypted variables that you can use in pipelines without exposing their value. Secret variables can be used for private information like passwords, IDs, and other identifying data that you wouldn’t want to have exposed in a pipeline. Secret variables are encrypted at rest with a 2048-bit RSA key and are available on the agent for tasks and scripts to use.

How to set Secret in Azure Variable group?

Set secret variables in the UI for a pipeline. Secret variables set in the pipeline settings UI for a pipeline are scoped to the pipeline where they are set. So, you can have secrets that only visible to users with access to that pipeline. Set secrets in a variable group. Variable groups follow the library security model. You can control who can define new items in a library, and who can use an existing item.

Let’s create a Secret variable in a Variable Group as shown below and make sure that you set it as a secret by locking it.

Once you mark it a secret (by clicking on the open lock icon as shown in below image), save the Variable Group, no one including admin will be able to view the secret. Let’s now understand how to view the secret with the help of Azure DevOps – Pipelines.

View the secret variables from Variable Group

You can create a simple Pipeline which has the below tasks to view the secrets in pipeline execution.

  1. PowerShell task which outputs a text (along with secret) into a file names ViewSecretValue.Txt
  2. Publish the ViewSecretValue.txt into Azure Pipeline artifacts.

Run the pipeline with with below PowerShell task

variables:
- group: Demo_VariableGroup
steps:
- powershell: |
    "The secretkey value is : $(secretkey)" | Out-File -FilePath  $(Build.ArtifactStagingDirectory)\ViewSecretValue.txt
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'drop'
    publishLocation: 'Container'

Now, click on the ViewSecretValue.txt file to download the file. Once you download it, view that in a Notepad which should below.

Conclusion

In summary, handling secret variables securely is crucial for maintaining data confidentiality in DevOps processes. Azure DevOps provides built-in features and best practices to keep sensitive data protected, making it a powerful platform for secure CI/CD pipeline management. Integrating with tools like Azure Key Vault can further strengthen your security posture and simplify secret management across multiple projects.

 

Devops

How to pass objects between tasks in Azure pipeline

Leave a comment

In our previous post, we already discussed about “How to pass values between Tasks in a Pipeline” where we can pass single value from one task/job/stage to another in pipeline. But incase, if you want to share the object from one task/job/stage to another instead of single value then we need to perform some trick to achieve this. In this quick post we will discuss about trick that I have found recently to store an object in an Azure Pipelines variable from a PowerShell script (How to pass objects between tasks in Azure pipeline).

The problem

Setting a variable in a script for later use in an Azure DevOps’ pipeline is possible using the task.setvariable command as described in below post.

How to pass values between Tasks in a Pipeline using task.setvariable Command

This works great for simple variables like below:

steps:
- pwsh: |
Write-Host "##vso[task.setvariable variable=variableName]variableValue"
- pwsh: |
Write-Host "Value from previous step: $(variableName)"

But it is bit trickier for sharing the complex variables likes objects, arrays, or arrays of objects between taks/stage/jobs in pipeline.

The solution

As an example, let’s we try to retrieve the name, type and resource group of all the resources in an Azure subscription as shown in the below script. Let we see how can pass this value of $resources in pipeline.

$azure_resources = Get-AzResource | Select-Object -Property Name,Type,ResourceGroupName

First you can store an object in an Azure Pipelines variable using the PowerShell task. Next, you can simply serialize it in JSON and apply in single input like below: -Compress flag which conver JSON to a single line.

$azure_resourcesJson = $azure_resources | ConvertTo-Json -Compress

Pass objects between tasks in Azure pipeline

pool:
name: devopsagent-win-pprd

steps:
- task: AzurePowerShell@5
inputs:
azureSubscription: 'Azure_Digital'
azurePowerShellVersion: LatestVersion
ScriptType: InlineScript
Inline: |
$azure_resources = Get-AzResource | Select-Object -Property Name,Type,ResourceGroupName -First 3
$azure_resourcesJson = $azure_resources | ConvertTo-Json -Compress
Write-Host "##vso[task.setvariable variable=resources]$azure_resourcesJson"
- pwsh: |
$resources = '$(resources)' | ConvertFrom-Json
Write-Host "There are $($resources.Count) resources in the list"
Write-Host "There are resources are" $resources.ResourceGroupName

OUTPUT