Connect Azure

The app registration with no secret, the federated credential, the custom reader role and its exact action list — rendered from the file that defines it.

Connecting a subscription

An Azure subscription is an account of its own cloud, registered by its id and read as an app registration in the customer's directory. Seven collectors read an enabled subscription once per cycle: virtual machines, AKS clusters, SQL databases, PostgreSQL flexible servers, storage accounts, virtual networks and App Service apps, each from one subscription-wide list (the capability matrix, "The Azure reader"), and Performance's perf.azuremonitor reads Azure Monitor's metrics of the measured ones once an hour, under the role's one action that is no collector's: Azure Monitor's metrics read. A statistic Azure refuses for a metric (InvalidSamplingType: an aggregation that metric does not serve) costs that series alone: the run stores every other series, moves on, and is finalized with the refused series named in its message on the collector matrix. Nothing for an operator to change; it names a catalogue entry to correct. A resource Azure Monitor answers "Failed to find metric configuration" for — a Linux consumption function app, at times — is dropped for that run alone: nothing of it is stored, every other resource is, and the finalized run names it with Azure's words; it is read again on the next run (slice 5). Any other 400 still fails the run. Azure answers zeros for the time before a resource existed, so no point is kept from before the inventory's creation time (a type's own, or Resource Manager's systemData.createdAt); App Service gives none, so a site's first pass starts at the first interval in which any of its series is not zero. work signs in as serve does, from the same AZURE_CERTIFICATE_FILE or none. Admin → Accounts offers Azure where this deployment reads it: the Azure block's Connect asks for the subscription, the directory and the app registration by their ids, and answers with the onboarding below; the API and the CLI register the same way. Everything here is accepted on fakes; it is proved live, against the test subscription.

A list Azure throttles is waited out as its Retry-After says, up to three times and thirty seconds a wait; one still throttled, or one whose x-ms-ratelimit-remaining-*-reads headers say the subscription's read budget is spent before its last page, ends the run incomplete, and the next cycle reads it again — what was seen is kept, and nothing is tombstoned on its strength.

The platform holds no secret for Azure. Its own AWS identity asks STS for a web identity token (AWS outbound identity federation), and Entra ID exchanges it for a Resource Manager token because the app trusts that token's issuer, subject and audience as a federated credential. The issuer and subject are the platform's, the same for every organisation it serves; the audience is the organisation's own, api://AzureADTokenExchange/<organisation id>, and the platform asks for no other when it reads that organisation's subscriptions. That is what an AWS account's per-tenant external id is: without it, one organisation could register another's app and subscription and the platform would read them for it. For an operator who knows AWS: a subscription is an account, the app registration is a role, the federated credential is its trust policy, and the custom role is a customer-managed policy.

Once per deployment, in the platform's AWS account (the owner's step on ours; prepared, not run):

aws iam enable-outbound-web-identity-federation
aws iam get-outbound-web-identity-federation-info   # the issuer, https://<id>.tokens.sts.global.api.aws

and the platform's role allowed to ask for the token, for Entra ID's audiences only, for at most five minutes:

{"Effect": "Allow", "Action": "sts:GetWebIdentityToken", "Resource": "*",
 "Condition": {"ForAllValues:StringLike": {"sts:IdentityTokenAudience": "api://AzureADTokenExchange/*"},
               "NumericLessThanEquals": {"sts:DurationSeconds": 300}}}

A platform role made by deploy/cloudformation/platform-identity.yaml gets exactly this statement from the stack's AzureFederation=true, off by default:

aws cloudformation update-stack --stack-name cloudpanorama-platform-identity \
  --template-body file://deploy/cloudformation/platform-identity.yaml --capabilities CAPABILITY_NAMED_IAM \
  --parameters ParameterKey=AzureFederation,ParameterValue=true <every other parameter: UsePreviousValue=true>

The token is asked of the region's STS endpoint (AWS_REGION), because the global endpoint does not serve it. The issuer and the subject the customer's app must trust are the ones in the platform's own token, and the onboarding reads them from one: they are the account's issuer and the platform role's ARN.

Under a Google home the same federated credential trusts the platform service account instead: issuer https://accounts.google.com, subject GCP_PLATFORM_SERVICE_ACCOUNT's numeric id, audience the organisation's api://AzureADTokenExchange/<organisation> — the onboarding reads the issuer and subject from the platform's own token, as it does here. A deployment whose home is Azure signs with its certificate, below.

A deployment off AWS has no such issuer, and signs with a certificate instead (AZURE_CERTIFICATE_FILE, a PEM file with the key and its certificate). The customer uploads the certificate to the app; the key never leaves the deployment. It is not a shared secret, but it is not federation either, and is for that case only. Its assertion is addressed to Entra ID's token endpoint and cannot name the organisation, so an app that holds the certificate trusts every organisation on that deployment: option B suits a deployment off AWS that serves one organisation. No path takes a client secret.

Per subscription, in the customer's directory and subscription — the template does all of it. It and the role file are in deploy/azure/, which the install copied out of the image (from platform 0.1.2; the README's Quickstart):

cd deploy/azure
az login --tenant TENANT_ID
az account set --subscription SUBSCRIPTION_ID
az deployment sub create --location westeurope --template-file reader.bicep \
  --parameters issuer=ISSUER subject=SUBJECT audience=AUDIENCE

or the same as commands (APP_ID is what az ad app create prints):

az ad app create --display-name panorama-reader --query appId -o tsv
az ad sp create --id APP_ID
az ad app federated-credential create --id APP_ID --parameters \
  '{"name":"panorama-federation","issuer":"ISSUER","subject":"SUBJECT","audiences":["AUDIENCE"]}'
sed s/SUBSCRIPTION_ID/<the subscription id>/ reader-role.json > role.json
az role definition create --role-definition @role.json
az role assignment create --assignee APP_ID --role "Panorama metadata reader" \
  --scope /subscriptions/SUBSCRIPTION_ID

The role is deploy/azure/reader-role.json, a custom role holding exactly the actions the platform's reads use — never the built-in Reader, whose */read reads app settings and Key Vault secrets' names. Then register it:

panorama admin add-azure-subscription --tenant-slug acme --account-id SUBSCRIPTION_ID \
  --name "Acme production" --azure-tenant-id TENANT_ID --client-id APP_ID

The registration is written to the audit log as estate.account.register, with the system as the actor.

ISSUER, SUBJECT and AUDIENCE are the onboarding's (GET /admin/accounts/{accountId}/onboarding, or the answer of POST /admin/accounts/subscriptions, which registers the same way), whose steps are these with all three filled in. Verification (panorama admin verify-account, or POST /admin/accounts/{accountId}/verify) signs in as the app — a refused exchange is reported with Entra ID's AADSTS code — reads the subscription, then the app's own permissions at the subscription: it fails when an action the reads need is not granted there (a role assigned at a resource group grants nothing at the subscription), and when the app holds any action beyond the reader role's — a wildcard such as the built-in Reader's */read, any data action, any */listKeys/action or */listSecrets/action, anything else — naming what a known one would hand over (internal/onboarding's never-granted list). A NotAction counts only in the role assignment it is in. A token the platform holds is sent to Resource Manager alone, over https, and a redirect is never followed. The account id is the subscription id, lower-cased; an ARM id is case-insensitive, and the key holds it lower-cased by resourcekey.FromAzureID alone.

What the reader role may do