In today’s cloud-centric world, monitoring Azure services and resources is paramount for maintaining the health, performance, and security of your cloud infrastructure. Azure provides a wide array of monitoring and logging capabilities, and one powerful tool in your monitoring arsenal is the Kusto Query Language (KQL).

In this blog, we’ll explore why KQL is essential for monitoring Azure services and resources and provide relevant code examples to help you get started.

Why is KQL Needed for Azure Monitoring?

Azure generates huge amounts of log and telemetry data across services and resources. This data holds valuable insights on the health and performance of your Azure environment. However, making sense of such large volumes of data is challenging.

KQL provides a powerful and flexible query language that can be used to analyze and derive insights from the massive amounts of log and telemetry data in Azure. KQL is a read-only language, so you don’t need to think about INSERT, DELETE, UPDATE and so on.

Check out this example to show all Azure blobs

// Show blob storage requests
StorageBlobLogs

Pretty amazing, huh! 

Here are some key reasons why KQL is crucial for Azure monitoring:

  • Fast and scalable: KQL is designed for massive data volumes. Queries execute quickly over terabytes of data. This enables analyzing data in near real-time.
  • Powerful analytics: KQL includes abilities like aggregation, joins, ranking, and more that make analysis easier. You can get started with simple queries and evolve to build advanced analytics.
  • Customizable: KQL is customizable to augment built-in functionality with user-defined functions, custom aggregations, and more.

1. Centralized Data Access

Azure services generate an abundance of logs, metrics, and telemetry data. KQL provides a unified language to query this data, enabling you to access information from various sources in one place.

KQL includes abilities like aggregation, joins, ranking, and more that make analysis easier. You can get started with simple queries and evolve to build advanced analytics.

2. Real-time Insights

KQL allows you to perform real-time analysis on your Azure resources. You can quickly identify and respond to issues, making it an invaluable tool for maintaining system health.

KQL is designed for massive data volumes. Queries execute quickly over terabytes of data. This enables analyzing data in near real-time.

3. Customized Analytics

KQL is incredibly flexible. You can write custom queries to gain deep insights into your specific Azure resources and applications, tailoring your monitoring to your organization’s unique needs.

KQL is customizable to augment built-in functionality with user-defined functions, custom aggregations, and more.

4. Unified query language

Azure Monitor uses KQL as its query language, making it a seamless choice for monitoring Azure resources. KQL provides a common syntax and semantics that works across various Azure data sources like Log Analytics, Application Insights, etc.

It’s tightly integrated with Azure services like Log Analytics, Application Insights, and Security Center. This avoids having to learn different query languages.

Now that we understand why KQL is essential, let’s explore some practical examples of how to utilize it effectively for monitoring Azure services and resources.

Real KQL Examples

Important thing to note in KQL is the | (pipe) element. KQL queries use pipe as symbol to delineate each operation. Additionally the operations are executed in the order they are written.

Simply put, the data set is manipulated at each step, and then the resulting set is “piped” to the next step. 

Example 1: Querying Azure Activity Logs

Azure Activity Logs record all operations performed on resources in your Azure subscription. You can use KQL to query these logs to track changes and audit activities. Here’s an example:

AzureActivity
| where ResourceGroup == "YourResourceGroup" and OperationName == "Microsoft.Compute/virtualMachines/delete"
| project Resource, Caller, ActivityStatus, ActivityDateTime
| order by ActivityDateTime desc

This query retrieves information about deleted virtual machines in a specific resource group, including the user who initiated the operation and the timestamp.

Example 2: Analyzing Azure Metrics

Azure Monitor Metrics provide data on the performance of your Azure resources. You can use KQL to analyze these metrics and create custom alerts. Here’s an example querying CPU usage for a virtual machine:

Perf
| where ObjectName == "Processor" and CounterName == "% Processor Time" and InstanceName == "_Total"
| summarize avg(CounterValue) by bin(TimeGenerated, 1h), Computer
| render timechart

This query plots a time chart of the average CPU usage for all virtual machines in your environment over the last hour.

Example 3: Identifying Security Threats

Azure Security Center generates security alerts and recommendations. You can leverage KQL to investigate and respond to security threats. Here’s a simple query to find high-severity security alerts:

SecurityAlert
| where Severity == "High"
| project TimeGenerated, AlertName, ResourceName, ResourceGroup, Status

This query lists high-severity security alerts, along with relevant information like the resource name and alert status.

Example 4: Query Application Insights data

app('myApplication').requests
| summarize requestCount=count(), durationAvg=avg(duration) by name
| order by requestCount desc

This query fetches HTTP request data from an Application Insights application, calculates the count and average duration for each request name and sorts the results by highest count.

Example 5: Analyze Azure activity logs

AzureActivity | 
where OperationName == "Restart Virtual Machine" | 
summarize count() by ResourceGroup, VirtualMachineName

This query processes Azure activity logs, filters log rows for virtual machine restart operations, and summarizes restart counts by resource group and VM.

Example 6: Monitor VM disk usage with Log Analytics

Perf 
| where ObjectName == "Logical Disk" and CounterName == "Free Megabytes" 
| summarize avg(CounterValue) by Computer, _ResourceId
| extend Disk_Used = 10000 - avg_CounterValue

This query calculates average free disk space from Perf table in Log Analytics workspace and derives used disk space.

Conclusion

Kusto Query Language (KQL) is an indispensable tool for monitoring Azure services and resources effectively. Whether you’re analyzing activity logs, performance metrics, or security alerts, KQL’s flexibility and power make it a go-to choice for gaining insights and maintaining the health and security of your Azure environment.

Start using KQL today to unlock the full potential of your Azure monitoring capabilities.

Further Reading:

Automating Cloud Computing Tasks with Ansible: Simplifying Infrastructure Management. 

Discover the key differences between a Full-Stack Developer and a Software Engineer in this blog.