How to visualize Azure Stack Hub Admin API Metrics in Grafana Dashboard – Part 2

Script logic

The 1st part of the blog post was about the prerequisites, the steps for the installation and some details you need to consider before you start with the project.
Now I will take a look into the script logic and how to get data from the Azure Stack Hub Admin API with powershell.

Input Parameters

First I define all input parameters needed. Normally I get the date, then I define the logpath and then I define every other parameter needed in the script.

#####################################
# Input Parameters                  #
#####################################
$date = Get-Date -Format "yyMMdd-HHmmss"
$logPath = "E:\Prometheus-powershell-Export\logs"
$location = "yourLocation"
Start-Transcript -Path $logPath\${date}-Get-AzureStack-$location-Admin.ps1.log -Force
$ArmEndpoint   = 'https://adminmanagement.$location.contoso.com'
$client_id     = 'your client id'
$client_secret = 'your client secret'
$AADTenantName = "yourAADTenant.onmicrosoft.com"

$EnvironmentName = "AzureStackAdmin$location"
$jobname = "azs-admin-metrics"
$stamp = $location 
$adminSubscription = "your admin subscription id"

Clean Logfiles

As my next step I like to delete old logfiles. I am a fan of a script execution history, but I also like to be organized and no one needs thousands of logfiles. For that reason I always clean the file system. In this example I clean all logfiles older than 7 days.

#####################################
# Clean old logfiles                #
#####################################
$logFile = "*-Get-AzureStack-$location-Admin.ps1.log"
$timeLimit = (Get-Date).AddDays(-7) 
if(Test-Path "$logPath\$logFile"){Get-ChildItem "$logPath\$logFile" -File | where { $_.LastWriteTime -lt $timeLimit } | Remove-Item -Force}

Set variables for Login

There is an awesome blog post from Robert van Vugt where he explains how to login to the Azure Stack Hub API and how to use different methods for the login.
http://robertvanvugt.com/how-to-call-azure-stack-rest-api-using-adfs-identity-provider/
Microsoft also has a very good documentation on how to connect with the API.


Here I choose to use a service principal in Azure AD.

############################################################
# Set variables to get the endpoint Information needed     #
# Get Tenant Id and create access token                    #
############################################################
 
$Environment = Add-AzureRmEnvironment -Name $EnvironmentName -ARMEndpoint $ArmEndpoint
$ActiveDirectoryServiceEndpointResourceId = $Environment.ActiveDirectoryServiceEndpointResourceId.TrimEnd('/')
$AuthEndpoint = (Get-AzureRmEnvironment -Name $EnvironmentName).ActiveDirectoryAuthority.TrimEnd('/')
$TenantId = (invoke-restmethod "$($AuthEndpoint)/$($AADTenantName)/.well-known/openid-configuration").issuer.TrimEnd('/').Split('/')[-1]
$AccessTokenUri = (invoke-restmethod "$($AuthEndpoint)/$($AADTenantName)/.well-known/openid-configuration").token_endpoint

Create Token to authenticate to the Web API

#####################################
# Request Bearer Token              #
#####################################

# Request Bearer Token
$body = "grant_type=client_credentials&client_id=$client_id&client_secret=$client_secret&resource=$ActiveDirectoryServiceEndpointResourceId"
$Token = Invoke-RestMethod -Method Post -Uri $AccessTokenUri -Body $body -ContentType 'application/x-www-form-urlencoded'

# Create Rest API Headers
$Headers = @{}
$Headers.Add("Authorization","$($Token.token_type)" + " " + "$($Token.access_token)")
$Headers.Add("Accept","application/json")
$Headers.Add("x-ms-effective-locale","en.en-us")

Get metrics from the API

Now everything is set to get the data from the Azure Stack Hub API!

# List Region Health through API
# Build the web request
$ListRegionHealth = "$ArmEndPoint/subscriptions/$adminSubscription/resourcegroups/system.local/providers/Microsoft.InfrastructureInsights.Admin/regionHealths?api-version=2016-05-01"
# Invoke the web request
$RegionHealth = (Invoke-RestMethod -Uri $ListRegionHealth -ContentType "application/json" -Headers $Headers -Method Get -Debug -Verbose).value

The variable “$RegionHealth” holds the data from the web request. Remember, that the data now needs to be transformed to the prometheus exposition format.

$results = $null
foreach($x in $RegionHealth)
    {
    foreach($metric in $x)
        {
        foreach($m in $metric.properties.usageMetrics)
            {

            $name = $m.name.Replace(" ","_").ToLower()
            $location = $x.location

            $result1 = "#HELP {0}`n" -f $name
            $result2 = "#TYPE {0} gauge`n" -f $name
            $results += $result1
            $results += $result2
            
            foreach($mValue in $m.metricsValue)
                {
                $valueName = $mValue.name
                $unit = $mValue.unit
                $value = $mValue.value

                $result3 = "azs_{0}{{location=`"{1}`",valueName=`"{2}`",unit=`"{3}`"}} {4}`n" -f @($name, $location, $valueName, $unit, $value)
                $results += $result3

                }

            }
        }
    }

# push the variable "$result" to the prometheus pushgateway
$responseRegionHealth = Invoke-WebRequest -Uri "http://localhost:9091/metrics/job/$jobname/instance/$stamp" -Method Post -Body $results

Output: prometheus exposition format

#HELP physical_memory
#TYPE physical_memory gauge
azs_physical_memory{location="yourLocation",valueName="Used",unit="GB"} 287
azs_physical_memory{location="yourLocation",valueName="Available",unit="GB"} 631
#HELP physical_storage
#TYPE physical_storage gauge
azs_physical_storage{location="yourLocation",valueName="Used",unit="TB"} 14.34
azs_physical_storage{location="yourLocation",valueName="Available",unit="TB"} 20
#HELP public_ip_address_pools
#TYPE public_ip_address_pools gauge
azs_public_ip_address_pools{location="yourLocation",valueName="Used",unit="One"} 39.0
azs_public_ip_address_pools{location="yourLocation",valueName="Available",unit="One"} 24.0

Get all the data you need!

Now you can have a look at the Azure Stack Hub Admin API reference documentation:
https://docs.microsoft.com/en-us/rest/api/azure-stack/

You can push any result you can get from the API to prometheus.

Next part will be my Dashboard and the API requests to fill it with data!!!