Azure Arc-Enabled Servers: Onboarding at Scale, Machine Configuration Guest Policy, and Extended Security Updates

A fleet of 600 Windows and Linux servers spread across two colo datacenters, an AWS account, and a GCP project is not a “we’ll get to it” problem. The moment your security team asks “which of these is missing a CIS baseline, which still runs an out-of-support OS, and who can touch them,” you need one control plane. Azure Arc-enabled servers projects each machine into Azure Resource Manager as a Microsoft.HybridCompute/machines resource. From there, the same management groups, Azure Policy assignments, RBAC, and Resource Graph queries you use for native Azure VMs reach into your hybrid estate — without lifting a single workload.

The pain this kills is governance fragmentation. Every server outside Azure today lives in a different tool: SCCM here, Ansible there, a spreadsheet of “boxes we should really patch.” There is no single answer to “is this fleet compliant,” no single RBAC plane, no single audit trail. Arc collapses that into ARM: one inventory, one policy engine, one identity model, one query language. The agent is read-mostly and outbound-only, so the security review is tractable; the per-machine cost for the management plane is zero (you pay only for the value-add services — Defender, Update Manager extras, ESU — you actually consume).

This walkthrough does the work a platform team actually has to do: onboard at scale non-interactively, manage extensions, enforce in-guest configuration with Machine Configuration (formerly Guest Configuration), report compliance through Azure Policy, deliver Extended Security Updates (ESU) for Windows Server 2012/2012 R2 through Arc, lock agent traffic behind a Private Link Scope, and scope RBAC so a stolen credential’s blast radius stays small. By the end you will be able to onboard a machine, prove it compliant, and explain every byte the agent puts on the wire. I assume Owner on the subscription and root/administrator on the machines.

What problem this solves

Hybrid estates rot in the dark. A server in a colo that nobody onboarded to any management plane is invisible: it does not appear in your secure-score, it does not get patched on a schedule, its drift from the CIS baseline is unknown, and the list of who can RDP to it lives in someone’s head. When the auditor or the breach forces the question, you have no answer and no tooling to get one fast. Multiply by 600 machines across three clouds and the gap is not an inconvenience — it is the finding that fails the audit.

What breaks without Arc: you run N management tools for N environments, each with its own identity model and its own blind spots. Patching is manual or scripted per-site, so a critical CVE sits unpatched on the boxes nobody remembered. Compliance is a quarterly fire drill of screenshots instead of a live Resource Graph query. Secrets sit on disk because there is no managed identity to authenticate scripts. And legacy Windows Server 2012/2012 R2 hosts — the ones you cannot migrate for 18 months — run with no security updates at all because ESU outside Azure requires a delivery channel you do not have.

Who hits this: every organization with servers outside Azure that still need Azure-grade governance — regulated enterprises (PCI, HIPAA), companies mid-migration with a long on-prem tail, and multicloud shops who refuse to run three separate governance stacks. The teams who feel it most acutely are the platform/SRE function asked to produce one compliance dashboard, and the security function that needs RBAC and audit to reach the boxes ARM cannot otherwise see.

To frame the whole field before the deep dive, here is every capability Arc adds, the on-prem pain it replaces, and where in this article it lives:

Learning objectives

By the end of this article you can:

• Explain the Connected Machine agent architecture — its services, its outbound-only 443 connectivity, the local IMDS endpoint, and the three connectivity modes — and validate reachability with azcmagent check before onboarding.
• Onboard hundreds of servers non-interactively with a least-privilege service principal, keep the secret off the command line and out of logs, and handle golden-image cloning without hostname collisions.
• Author, sign, and publish a Machine Configuration package, assign it through Azure Policy, and choose correctly between Audit, ApplyAndMonitor, and ApplyAndAutoCorrect.
• Drive Azure Policy guest assignments at management-group scope, run remediation tasks against a pre-existing fleet, and answer “what’s non-compliant” with one Resource Graph query.
• Provision and link Extended Security Updates licenses for Windows Server 2012/2012 R2 with the correct Type/Edition/Processors, manage them as code, and stop billing cleanly on decommission.
• Stand up an Azure Arc Private Link Scope with the right private DNS zones, and identify the two endpoints (Entra ID, ARM) that never traverse the scope.
• Scope RBAC with purpose-built Arc roles instead of Contributor, layer deny policy guardrails on extensions and tags, and reason about blast radius.
• Run a symptom → cause → confirm → fix playbook for the agent failures you will actually hit in production.

Prerequisites & where this fits

You should be comfortable with the Azure governance fundamentals: how management groups, subscriptions, and resource groups nest (see Azure Resource Hierarchy Explained), how Azure Policy definitions, initiatives, assignments, and effects work (see Azure Policy: Governance at Scale), and how RBAC role assignments scope down (see Azure Entra RBAC Governance Deep Dive). You should be able to run az in Cloud Shell, read JSON output, and write a small Bicep template. Familiarity with managed identities (Entra Managed Identities Deep Dive) and Private Link / private DNS (Azure Private Link & Private DNS for PaaS) will make the networking sections land faster.

This sits in the Hybrid & Governance track. Arc-enabled servers is the foundation; its siblings extend the same control plane to other resource types — Azure Arc-Enabled Kubernetes: GitOps, Policy & Fleet Management does for clusters what this article does for VMs. Downstream of onboarding sit the value-add services: Azure Update Manager: Maintenance Configurations & Patch Orchestration with Arc for fleet patching, Defender for Servers for threat protection, and the Azure Monitor data-collection pipeline for telemetry. In a full platform this all lands inside an Enterprise-Scale Landing Zone management-group hierarchy.

A quick map of who owns what during a rollout, so you assign the work correctly:

Core concepts

Five mental models make every later decision obvious.

Arc projects a server into ARM as a resource you can govern. A connected machine becomes a Microsoft.HybridCompute/machines resource with the same primitives as a native VM: tags, RBAC at the resource scope, policy assignments inherited from its resource group / subscription / management group, and a row in Resource Graph. Nothing about the workload changes — the OS, the apps, the network all stay put. What changes is that Azure’s governance plane now reaches the box. The mental shift: Arc is not a migration and not an agent that runs your workload; it is a control-plane projection.

The agent is low-privilege, outbound-only, and identity-bearing. The Connected Machine agent is one package — the azcmagent CLI plus three services (the Hybrid Instance Metadata Service, the GuestConfig service, and the extension manager). It runs as a low-privilege service, polls Azure over outbound HTTPS (443) only, and never opens an inbound port. On connect, the machine receives a system-assigned managed identity backed by a certificate the agent rotates; in-guest tooling reads tokens from a local IMDS endpoint at http://localhost:40342, so scripts authenticate with no stored secret.

Machine Configuration is the in-guest enforcement engine, and it is in-box. Azure Policy alone sees ARM-level properties; it cannot see a registry value or a sysctl setting inside the OS. Machine Configuration runs DSC-style packages inside the guest to assert that state — registry keys, file contents, installed packages, service states, secedit/sysctl. Critically, on Arc servers the engine ships in the agent — you do not deploy a separate ConfigurationforWindows/ConfigurationforLinux extension as you do on Azure VMs. That removes a whole DeployIfNotExists prerequisite from your design.

Connectivity has three modes, and two endpoints always go public. The agent connects in direct mode (outbound to public endpoints, optionally via proxy), proxy mode, or Private Link mode (the his and guestconfiguration data planes over a private endpoint). The trap that sinks rollouts: Entra ID (login.microsoftonline.com) and Azure Resource Manager (management.azure.com) traffic always use public endpoints, even with Private Link. Block them and the agent cannot authenticate no matter how healthy the private endpoint is.

Governance reaches in: policy, ESU, and patching all ride the projection. Once a machine is in ARM, the value flows: assign baselines via policy at management-group scope and they cascade; deliver Extended Security Updates to off-Azure WS2012/2012 R2 boxes through a license resource you link to the machine; orchestrate patches across the whole fleet with Update Manager; reach the box with az ssh arc through ARM with no inbound port. The single projection is the foundation every other capability stands on.

The vocabulary in one table

Pin down every moving part before the deep sections. The glossary at the end repeats these for lookup; this table is the model side by side:

1. The Connected Machine agent: architecture and connectivity modes

The agent is a single package — the azcmagent CLI plus services (the Hybrid Instance Metadata Service, the GuestConfig service, the extension manager). It runs low-privilege, polls Azure over outbound HTTPS (443) only, and never opens an inbound port. Three facts shape every design decision:

• Identity. On connect, the machine gets a system-assigned managed identity backed by a certificate the agent rotates. In-guest tooling reads tokens from a local IMDS endpoint (http://localhost:40342), so scripts authenticate with no stored secret.
• Machine Configuration is built in. Unlike Azure VMs, Arc servers do not need the ConfigurationforWindows/ConfigurationforLinux extension deployed separately — the agent ships it in-box, removing a whole DeployIfNotExists step from your design.
• Connectivity modes: direct (outbound to public endpoints, optionally via proxy), proxy, and Private Link (his/guestconfiguration data-plane over a private endpoint). Entra ID and Resource Manager traffic always use public endpoints even with Private Link — plan firewall rules accordingly.

The agent’s moving parts, what each does, and what it talks to:

The three connectivity modes, side by side — pick per security posture:

Every endpoint the agent needs, the mode that uses it, and what breaks if you block it:

Configure the proxy before connecting so onboarding itself can route out:

azcmagent config set proxy.url "http://proxy.corp.local:3128"
azcmagent config set proxy.bypass "Arc,ArcData"   # built-in bypass lists

# Verify reachability of every required endpoint BEFORE onboarding
azcmagent check --location eastus

azcmagent check returns a pass/fail per required FQDN (*.his.arc.azure.com, *.guestconfiguration.azure.com, login.microsoftonline.com, management.azure.com, and the download CDN). Bake it into golden-image validation.

The azcmagent subcommands you will actually use, and when:

Two firewall facts that catch teams every single time:

2. At-scale onboarding with a service principal

Interactive login does not scale to 600 servers. Create a dedicated onboarding service principal with the narrowest role for the job — Azure Connected Machine Onboarding — scoped to the single resource group that holds the machines. It can create Arc server resources and nothing else; a leaked secret cannot pivot.

# Dedicated onboarding identity, scoped to one RG, narrowest built-in role
az ad sp create-for-rbac \
  --name "sp-arc-onboarding" \
  --role "Azure Connected Machine Onboarding" \
  --scopes "/subscriptions/<sub-id>/resourceGroups/rg-arc-servers"

Never put the secret on a command line — azcmagent echoes arguments to logs in some failure paths. Use a config file referenced with --config; the agent reads the credential from disk and keeps it out of the console:

# /etc/arc-onboard.json  (mode 0600, deleted after onboarding)
cat > /etc/arc-onboard.json <<'JSON'
{
  "subscriptionId": "<sub-id>",
  "resourceGroup": "rg-arc-servers",
  "location": "eastus",
  "tenantId": "<tenant-id>",
  "servicePrincipalId": "<app-id>",
  "servicePrincipalSecret": "<secret>",
  "cloud": "AzureCloud"
}
JSON
chmod 600 /etc/arc-onboard.json

azcmagent connect \
  --config /etc/arc-onboard.json \
  --tags "Datacenter=COLO1,App=Payments,Owner='Platform Eng'" \
  --correlation-id "$(uuidgen)"

shred -u /etc/arc-onboard.json   # remove the secret immediately

Windows uses the same azcmagent connect --config from an elevated session. For golden images, do not onboard before cloning — install the agent, leave it disconnected, and let first-boot automation (cloud-init, Ansible, an MDT/Intune task) run connect with a per-machine --resource-name so hostnames do not collide. Certificate-based SPN auth (--service-principal-cert) is better where you can distribute certs — it removes the long-lived secret entirely. Use --use-azcli (agent 1.59+) only for ad-hoc operator onboarding, never unattended fleets.

The onboarding methods, ranked by fleet-fit:

The azcmagent connect flags that matter at scale, and why:

The least-privilege role math — what each onboarding-related role can and cannot do:

A pre-onboarding readiness checklist, as a table you can tick:

3. Machine Configuration: audit and remediation in-guest

Machine Configuration runs DSC-style packages inside the OS to assert state Azure Policy alone cannot see — registry values, file contents, installed packages, service states, sysctl/secedit settings. The engine is in-box on Arc servers, so you only assign policy.

Every configuration is an MOF compiled into a signed .zip package published to Blob Storage, then referenced by a policy definition. Author it with the GuestConfiguration PowerShell module:

Install-Module -Name GuestConfiguration -Scope CurrentUser

# Compile your DSC config (here: assert a registry value) then package it
New-GuestConfigurationPackage `
  -Name 'EnforceTlsRegistry' `
  -Configuration './EnforceTlsRegistry.mof' `
  -Type 'ApplyAndAutoCorrect' `   # Audit | ApplyAndMonitor | ApplyAndAutoCorrect
  -Path './package'

# Test against the local machine before publishing
Get-GuestConfigurationPackageComplianceStatus `
  -Path './package/EnforceTlsRegistry.zip'

The -Type you compile in determines behavior and maps directly to the assignmentType on the resulting guest assignment resource:

A subtlety that bites people: when a custom policy first deploys an assignment, assignmentType can briefly read Null before resolving (typically within an hour). Do not alert on that transient state.

Generate a policy definition from the package and assign it. For audit-only baselines (the CIS/STIG built-ins), the initiatives already exist — assign those directly rather than authoring your own.

New-GuestConfigurationPolicy `
  -PolicyId (New-Guid) `
  -ContentUri 'https://stgarc.blob.core.windows.net/pkgs/EnforceTlsRegistry.zip' `
  -DisplayName 'Enforce TLS registry baseline' `
  -Platform 'Windows' `
  -PolicyVersion '1.0.0' `
  -Mode 'ApplyAndAutoCorrect' `
  -Path './policy'

What Machine Configuration can assert (and what it cannot)

The engine reaches deep into the OS, but it is not a general-purpose config-management tool. Know the boundary:

The authoring-to-enforcement pipeline, stage by stage:

Built-in initiatives you should assign rather than author from scratch:

Common authoring mistakes and their fix:

4. Extension management and VM-like operations

Arc servers accept the same extension model as Azure VMs through Microsoft.HybridCompute/machines/extensions. Day one you deploy the Azure Monitor Agent (telemetry to a Data Collection Rule) and, where used, the Custom Script Extension. Push them at scale with policy, or imperatively for a single box:

# Install the Azure Monitor Agent extension on an Arc server
az connectedmachine extension create \
  --resource-group "rg-arc-servers" \
  --machine-name "colo1-pay-01" \
  --name "AzureMonitorWindowsAgent" \
  --publisher "Microsoft.Azure.Monitor" \
  --type "AzureMonitorWindowsAgent" \
  --enable-auto-upgrade true

--enable-auto-upgrade true opts the extension into automatic minor-version upgrades — set it everywhere so you are not chasing CVEs in the agents themselves. Keep the agent current too via automatic agent upgrade so azcmagent self-updates.

Beyond extensions, Arc unlocks VM-like operations: SSH/RDP over Arc (az ssh arc, through ARM with no inbound port), Azure Update Manager for cross-fleet patch orchestration, and Run Command for one-off scripts audited through ARM — replacing bastion and jump-box sprawl with RBAC-governed, logged access.

The extensions you will actually deploy on Arc servers, and what each delivers:

VM-like operations Arc unlocks, and what they replace:

Extension lifecycle states and what each means:

5. Azure Policy guest assignments and compliance reporting

The standard path is the built-in initiative “Deploy prerequisites to enable Guest Configuration policies on virtual machines.” On Azure VMs it deploys the extension and a system-assigned identity; on Arc servers the extension half is a no-op (it’s in-box) but the identity wiring still applies. Assign it at the management-group level so new machines inherit it.

DeployIfNotExists and Modify assignments act only on new or updated resources. To bring the existing 600 into compliance you must create a remediation task — the single most common reason teams see “0% compliant” and panic. Trigger it on the assignment:

# Remediate all existing in-scope machines for one policy assignment
az policy remediation create \
  --name "remediate-machinecfg-baseline" \
  --policy-assignment "<assignment-id>" \
  --resource-discovery-mode ReEvaluateCompliance

Compliance lands in Azure Resource Graph — how you answer “what’s broken” across the whole fleet in one query instead of clicking through the portal:

// Non-compliant Machine Configuration assignments across the estate
guestconfigurationresources
| where type =~ "microsoft.guestconfiguration/guestconfigurationassignments"
| extend status = tostring(properties.complianceStatus)
| extend machine = tostring(split(id, "/")[8])
| where status =~ "NonCompliant"
| project machine, name, status, lastComplianceChecked = properties.lastComplianceStatusChecked
| order by machine asc

Policy effects you will combine for an Arc estate, and what each does:

The complianceStatus values and how to read them:

The “0% compliant and panicking” decision table:

Scopes and inheritance — assign high, let it cascade:

6. Extended Security Updates for Windows Server 2012/2012 R2 through Arc

This is frequently the business case that funds the entire rollout. Windows Server 2012/2012 R2 are out of support; ESU delivers patches for up to three more years, and Arc is the delivery mechanism for machines not in Azure. You provision a license resource, then link it to each eligible server; patches flow through Windows Update / Azure Update Manager and bill monthly — no MAK keys to distribute.

The license is a first-class ARM resource (Microsoft.HybridCompute/licenses). Provision it with the CLI, attesting to Software Assurance or SPLA coverage:

# Provision a Datacenter physical-core ESU license (min 16 physical cores)
az connectedmachine license create \
  --license-name "esu-ws2012-dc-colo1" \
  --resource-group "rg-arc-servers" \
  --location "eastus" \
  --license-type "ESU" \
  --state "Activated" \
  --target "Windows Server 2012 R2" \
  --edition "Datacenter" \
  --type "pCore" \
  --processors 16

Watch the licensing rules that actually cost money:

• Type is pCore or vCore. Physical-core licenses carry a mandatory 16-core minimum; virtual-core a minimum of 8 per VM. The three valid combinations are Standard vCore, Standard pCore, and Datacenter pCore.
• You can resize --processors after provisioning and move licenses between resource groups/subscriptions — they are normal ARM resources, queryable in Resource Graph.
• Attesting to SA/SPLA coverage is a licensing commitment, not a checkbox.

The license parameters and their rules — get these wrong and you over- or under-pay:

The three valid license combinations (anything else is invalid):

Linking is a licenseProfiles/default child on the machine. Declare it in Bicep so it lives in source control alongside the license:

@description('Resource ID of the ESU license to assign')
param esuLicenseId string
param machineName string

resource esuLink 'Microsoft.HybridCompute/machines/licenseProfiles@2023-06-20-preview' = {
  name: '${machineName}/default'
  location: resourceGroup().location
  properties: {
    esuProfile: {
      assignedLicense: esuLicenseId
    }
  }
}

To unlink (machine decommissioned, or moved into Azure where ESU is free), PUT the same licenseProfiles/default with an empty esuProfile: {}. Deactivate a license by PATCHing its state to Deactivated so billing stops.

The ESU lifecycle, operation by operation:

ESU eligibility and “do I even need it” decision table:

7. Private Link Scope for secure agent-to-Azure traffic

For regulated estates that forbid agent traffic over the public internet, an Azure Arc Private Link Scope (Microsoft.HybridCompute/privateLinkScopes) routes the his and guestconfiguration data planes through one private endpoint over ExpressRoute or VPN. One scope serves many machines; a virtual network maps to at most one scope.

# Create the scope, then a private endpoint bound to its 'hybridcompute' group
az connectedmachine private-link-scope create \
  --resource-group "rg-arc-net" \
  --location "eastus" \
  --scope-name "pls-arc-prod" \
  --public-network-access Disabled

scopeId=$(az connectedmachine private-link-scope show \
  --resource-group "rg-arc-net" --scope-name "pls-arc-prod" --query id -o tsv)

az network private-endpoint create \
  --resource-group "rg-arc-net" \
  --name "pe-arc-prod" \
  --location "eastus" \
  --vnet-name "vnet-hub" \
  --subnet "snet-pe" \
  --private-connection-resource-id "$scopeId" \
  --group-id "hybridcompute" \
  --connection-name "arc-conn"

Two private DNS zones must resolve to the endpoint’s private IPs (a third only if you also run Arc-enabled Kubernetes):

privatelink.his.arc.azure.com
privatelink.guestconfiguration.azure.com
# privatelink.dp.kubernetesconfiguration.azure.com   # only for Arc K8s

Critically, Microsoft Entra ID (login.microsoftonline.com, pas.windows.net) and Azure Resource Manager (management.azure.com) do not traverse the scope — they keep using public endpoints. Allow those via the AzureActiveDirectory and AzureResourceManager service tags on your firewall/NSG, or servers fail to authenticate even with a healthy private endpoint. Onboard new machines with --private-link-scope <scope-resource-id>; associate existing ones afterward (up to 15 minutes to start accepting connections).

What goes private versus what stays public — the table that prevents the #1 Private Link failure:

Private DNS zones required, keyed by what you run:

Private Link Scope constraints worth internalizing:

8. RBAC scoping and operational guardrails

The whole point of projecting servers into ARM is that existing governance applies. Use the purpose-built Arc roles instead of broad Contributor:

Layer policy guardrails on top so the estate stays inside the rails:

• A deny on Microsoft.HybridCompute/machines/extensions restricted to an allowlist of publishers/types — extensions run as root/SYSTEM, so stop arbitrary ones from being pushed.
• A deny on machines missing required tags (Owner, Datacenter, DataClassification) so nothing onboards anonymously.
• Diagnostic settings shipping agent and policy events to a central Log Analytics workspace.

// Deny any Arc extension not on the allowlist (policyRule fragment)
"if": {
  "allOf": [
    { "field": "type", "equals": "Microsoft.HybridCompute/machines/extensions" },
    { "not": {
        "field": "Microsoft.HybridCompute/machines/extensions/type",
        "in": ["AzureMonitorWindowsAgent", "AzureMonitorLinuxAgent", "CustomScriptExtension"]
    }}
  ]
},
"then": { "effect": "deny" }

The guardrail policies every Arc estate should carry, and the blast radius each contains:

Blast-radius reasoning — what a stolen credential can do, by identity:

Architecture at a glance

Read the diagram left to right as the control-plane projection it is. On the far left, an off-Azure server in a colo or another cloud runs the Connected Machine agent — azcmagent plus the HIMDS, GuestConfig, and extension-manager services — listening on no inbound port and reaching out only on HTTPS 443. That outbound traffic splits at the connectivity layer: with Private Link, the his and guestconfiguration data planes ride a private endpoint through your hub VNet over ExpressRoute/VPN, while Entra ID and Azure Resource Manager stay on public endpoints (the rule that trips everyone — allow the AzureActiveDirectory and AzureResourceManager service tags or nothing authenticates). Once through, the agent authenticates to Entra ID, the machine surfaces in ARM as a HybridCompute/machines resource with a system-assigned managed identity, and from there the governance plane takes over.

The right of the diagram is where the value lands. Azure Policy assigns CIS/STIG baselines and Machine Configuration packages that the in-guest engine evaluates and (optionally) auto-corrects; ESU licenses link to each WS2012/2012 R2 box to fund off-Azure patching through Update Manager; and Resource Graph + Log Analytics roll the whole fleet’s compliance and telemetry into one queryable plane. The numbered badges mark the five places this breaks in production — onboarding egress, the always-public Entra ID/ARM path, the in-box Machine Config engine, the ESU link, and the private-endpoint DNS — and the legend narrates each as symptom · confirm · fix. Follow the path once and you have the whole system: agent out on 443, identity to Entra ID, resource in ARM, governance reaching back in.

Real-world scenario

A payments platform team I worked with — call them NorthPay — ran 420 Windows Server 2012 R2 hosts across two PCI-scoped datacenters. The constraint was hard: the auditor would not accept agent telemetry crossing the public internet, and every box needed ESU because migrating the legacy payment gateway off 2012 R2 was an 18-month project they could not front-load. Their first attempt onboarded everything in direct mode and immediately failed the network review.

The fix had three parts. First, a Private Link Scope per datacenter, fronted by a private endpoint on the existing ExpressRoute-connected hub VNet, with the his and guestconfiguration zones in central private DNS. Second — the part that broke — they had blocked all outbound internet at the firewall, and onboarding hung. The agent still needs Entra ID and ARM over the public internet even behind Private Link, so they added exactly two service-tag rules and nothing else:

# The only public egress the agent needs behind Private Link
az network nsg rule create -g rg-pci-net --nsg-name nsg-arc \
  --name AllowAAD --priority 150 --direction Outbound --access Allow \
  --protocol Tcp --source-address-prefixes VirtualNetwork \
  --destination-address-prefixes AzureActiveDirectory --destination-port-ranges 443
az network nsg rule create -g rg-pci-net --nsg-name nsg-arc \
  --name AllowARM --priority 151 --direction Outbound --access Allow \
  --protocol Tcp --source-address-prefixes VirtualNetwork \
  --destination-address-prefixes AzureResourceManager --destination-port-ranges 443

Third, ESU as code: one Datacenter pCore license per physical host (2-socket boxes, well over the 16-core floor), provisioned and linked through a Bicep loop over an inventory file with assignedLicense referencing the license resource ID. Compliance — the CIS baseline via Machine Configuration ApplyAndMonitor and ESU coverage — rolled up into one Resource Graph dashboard the auditor could query directly. The review passed on the second pass, and the only public traffic on the wire was two service tags to identity and ARM.

The numbers told the story to the CFO. The management plane itself cost ₹0 per machine; the spend was ESU (the unavoidable cost of running an out-of-support OS for 18 more months) plus a modest Log Analytics ingestion bill and the private-endpoint hours. Against the alternative — a forced, rushed migration of the payment gateway, or a failed PCI audit — it was trivially justified. The lesson NorthPay wrote on the wall: “Private Link makes the data plane private; it does not make identity private. Allow Entra ID and ARM, or you have a beautiful endpoint nobody can authenticate through.”

The rollout as a timeline, because the order of moves is the lesson:

Advantages and disadvantages

Projecting servers into ARM is powerful, but it is not free of trade-offs. Weigh it honestly:

The model is right when you have servers outside Azure that genuinely need Azure-grade governance, identity, and patching — regulated estates, long migration tails, multicloud shops. It is over-engineering for a handful of boxes you will retire next quarter, or for workloads that have no compliance, identity, or patching requirement at all. The disadvantages are all manageable — but only if you know they exist, which is the point of this article.

Hands-on lab

Onboard a single machine, prove it healthy, assign an audit baseline, and tear it down — all on a free-tier-friendly Linux VM (you can use a small Azure VM as the “off-Azure” stand-in, or any Ubuntu box you control). Run the Azure-side commands in Cloud Shell (Bash); run the agent commands on the target machine.

Step 1 — Variables and resource group.

RG=rg-arc-lab
LOC=eastus
SP_NAME=sp-arc-lab-onboard
az group create -n $RG -l $LOC -o table

Expected: a resource-group row with provisioningState: Succeeded.

Step 2 — Create the least-privilege onboarding SPN.

az ad sp create-for-rbac \
  --name "$SP_NAME" \
  --role "Azure Connected Machine Onboarding" \
  --scopes "$(az group show -n $RG --query id -o tsv)"
# Note the appId, password, tenant — you'll put them in a 0600 file on the box

Expected: JSON with appId, password, tenant. Treat the password like a secret.

Step 3 — On the target machine, install the agent. (Linux one-liner; Windows uses the MSI.)

# On the Ubuntu box (run as root)
wget https://aka.ms/azcmagent -O ~/install_linux_azcmagent.sh
bash ~/install_linux_azcmagent.sh
azcmagent version   # confirm the CLI is installed

Step 4 — Pre-flight the endpoints before connecting.

azcmagent check --location eastus
# Expect PASS for his, guestconfiguration, login.microsoftonline.com, management.azure.com, CDN

If any FQDN fails, fix egress before continuing — onboarding will hang otherwise.

Step 5 — Connect using the SPN via a 0600 config file (secret off the CLI).

cat > /etc/arc-onboard.json <<JSON
{ "subscriptionId":"<sub-id>","resourceGroup":"rg-arc-lab","location":"eastus",
  "tenantId":"<tenant>","servicePrincipalId":"<appId>","servicePrincipalSecret":"<password>",
  "cloud":"AzureCloud" }
JSON
chmod 600 /etc/arc-onboard.json

azcmagent connect --config /etc/arc-onboard.json \
  --tags "Owner=Lab,Datacenter=LAB,DataClassification=None" \
  --correlation-id "$(uuidgen)"

shred -u /etc/arc-onboard.json

Expected: Connected machine to Azure. The box now exists in ARM.

Step 6 — Verify health from both sides.

azcmagent show   # on the box: Status: Connected, an Agent Version, a MI principal id

# From Cloud Shell
az connectedmachine show -g $RG -n "$(hostname)" \
  --query "{status:status, agentVersion:agentVersion, mi:identity.principalId}" -o jsonc

Expected: status: Connected, a non-null mi.

Step 7 — Assign an audit-only CIS-style baseline at the RG scope. Use a built-in audit initiative so there’s nothing to author:

# Example: assign a built-in 'audit insecure password settings' style policy at the RG
az policy assignment create \
  --name "lab-audit-baseline" \
  --scope "$(az group show -n $RG --query id -o tsv)" \
  --policy-set-definition "<built-in-initiative-id>"   # pick an Arc-applicable audit initiative

Compliance takes time to evaluate; check it in Resource Graph after ~30–60 minutes with the guestconfigurationresources query from section 5.

Step 8 — Teardown (stop all billing and remove the resource).

# On the box: cleanly disconnect (removes the ARM resource + MI)
azcmagent disconnect --config /dev/null 2>/dev/null || azcmagent disconnect

# From Cloud Shell: nuke the RG and the SPN
az group delete -n $RG --yes --no-wait
az ad sp delete --id "<appId>"

Expected: the machine disappears from ARM; the RG deletes asynchronously. Nothing here incurs ongoing cost once removed.

Common mistakes & troubleshooting

Most Arc incidents are one of a dozen failure modes, and each has a precise signal. Scan the playbook, then read the detail for the row that matches.

Onboarding hangs (rows 1–3)

By far the most common rollout failure, and almost always egress. The agent must reach five endpoint classes; block any and connect stalls. The decision table:

Compliance shows 0% (rows 4–6)

The panic moment. Nine times in ten it is a remediation task that never ran, because DeployIfNotExists and Modify only act on new/updated resources:

ESU surprises (rows 7, 8, 14)

ESU is the part that touches the invoice, so its failure modes cost money, not just availability:

Best practices

• Onboard with a dedicated, RG-scoped SPN carrying only Azure Connected Machine Onboarding; prefer certificate auth, pass any secret via a 0600 --config file, and shred it after.
• Bake azcmagent check into golden-image validation. Never ship an image that can’t pre-flight its endpoints; configure the proxy before connect.
• Never pre-connect a golden image. Install the agent disconnected, and onboard at first boot with a per-machine --resource-name to avoid name collisions.
• Tag at onboard (Owner, Datacenter, DataClassification) and enforce a deny-untagged policy so nothing onboards anonymously.
• Assign policy at the management-group scope so new machines inherit baselines automatically; reserve resource-scope assignments for genuine exceptions.
• Always run a remediation task after any DeployIfNotExists/Modify assignment against a pre-existing fleet — assignment alone never touches existing machines.
• Pick assignmentType deliberately: Audit for reporting, ApplyAndMonitor for one-time enforcement, ApplyAndAutoCorrect only where self-healing is genuinely wanted.
• Restrict extensions to an allowlist with a deny policy — extensions run as root/SYSTEM and are a real attack surface.
• Enable auto-upgrade on both the agent and every extension so you are not chasing CVEs in your own tooling.
• Treat ESU licenses as code (Bicep), with the correct edition×type combo and core count; deactivate on decommission so billing stops.
• For regulated estates, use a Private Link Scope per hub VNet — and always allow the AzureActiveDirectory + AzureResourceManager service tags for the unavoidable public egress.
• Ship agent and policy diagnostics to a central Log Analytics workspace and build the compliance view in Resource Graph, not portal screenshots.

Security notes

The security posture of an Arc estate rests on three pillars: identity blast radius, the in-guest attack surface, and network exposure. Tighten each deliberately.

Identity-specific guidance:

The network exposure model in one line per layer: inbound — nothing, ever (the agent opens no port); outbound — 443 only, to a known FQDN set, ideally split into private (data plane) and tightly-scoped public (identity/ARM); lateral — a compromised box’s MI can do only what you granted it, so least-privilege the MI as if it were a user.

Cost & sizing

The headline that funds the rollout: the Arc management plane is free. There is no per-machine charge to project a server into ARM, run Machine Configuration audits, assign policy, or query Resource Graph. You pay only for value-add services you opt into. Knowing exactly what bills prevents both sticker shock and the opposite error — assuming Arc itself costs money and under-deploying.

ESU sizing is where the real money is, and it scales by cores, not machines:

Right-sizing rules of thumb:

The free tier in practice: onboarding, inventory, policy, Machine Configuration, and Resource Graph cost nothing — you can govern a 600-machine estate’s compliance for ₹0. The bill arrives only with ingestion (control your DCR scope), Defender (opt in where threat protection is needed), Update Manager extras, ESU (the cost of running an out-of-support OS), and private endpoints. Budget those five line items, not “Arc.”

Interview & exam questions

These map to AZ-104, AZ-500, and AZ-305, where hybrid governance, Arc, and Machine Configuration appear.

1. What does Azure Arc-enabled servers actually do to an on-prem machine? It projects the machine into Azure Resource Manager as a Microsoft.HybridCompute/machines resource, so ARM governance — RBAC, Azure Policy, Resource Graph, tags — reaches it. The workload, OS, and network are unchanged; only the control plane extends.

2. The agent is connected but in-guest scripts can’t get a token. Likely cause? The his data plane (*.his.arc.azure.com) is unreachable, so the Hybrid IMDS at localhost:40342 can’t broker tokens. Confirm with azcmagent show (null MI) and fix the egress or the private DNS zone.

3. You enabled a Private Link Scope but onboarding still fails to authenticate. Why? Entra ID (login.microsoftonline.com) and ARM (management.azure.com) always use public endpoints even with Private Link. You must allow the AzureActiveDirectory and AzureResourceManager service tags; the private endpoint only carries the his/guestconfiguration data planes.

4. Difference between Audit, ApplyAndMonitor, and ApplyAndAutoCorrect? Audit reports non-compliance and changes nothing. ApplyAndMonitor applies once at assignment then only reports drift. ApplyAndAutoCorrect runs the Set to remediate on every evaluation — continuous self-healing.

5. You assigned a DeployIfNotExists policy but the existing fleet shows 0% compliant. Fix? DINE and Modify act only on new/updated resources. Create a remediation task (az policy remediation create) to bring existing machines into scope.

6. Why don’t Arc servers need the Guest Configuration extension? The Machine Configuration engine ships in-box with the Connected Machine agent. On Azure VMs you deploy the extension separately; on Arc that half of the prerequisites initiative is a no-op (the identity wiring still applies).

7. What’s the minimum core count for an ESU pCore license, and which edition pairs with it? 16 physical cores minimum for pCore. Datacenter is only valid with pCore; the three valid combos are Standard vCore (min 8), Standard pCore (min 16), and Datacenter pCore (min 16).

8. How do you onboard 500 servers non-interactively without leaking a secret? A dedicated SPN with Azure Connected Machine Onboarding scoped to one RG; pass the secret via a 0600 --config file (never the CLI, which can echo to logs) and shred it after; prefer certificate auth where you can distribute certs.

9. A cloned golden image produced colliding machine names in ARM. What did they do wrong? They onboarded before cloning. Install the agent disconnected in the image, and run connect at first boot with a per-machine --resource-name.

10. Which role should the onboarding identity have, and why not Contributor? Azure Connected Machine Onboarding — it can only create/read Arc machine resources. Contributor would let a leaked secret manage extensions (root/SYSTEM code), ESU, and other resources, a far larger blast radius.

11. How do you stop ESU billing for a decommissioned server? Unlink it (PUT licenseProfiles/default with an empty esuProfile: {}) and, if no other machine uses the license, PATCH the license state to Deactivated to stop billing.

12. What is the single most common reason a Private Link Arc rollout fails the first time? Blocking all outbound internet, forgetting that Entra ID and ARM must stay public. The endpoint looks healthy but nothing can authenticate until the two service tags are allowed.

Quick check

1. Which two endpoint classes always use public endpoints, even behind an Arc Private Link Scope?
2. You see all existing machines as NonCompliant right after assigning a DeployIfNotExists policy. What did you forget?
3. What is the mandatory minimum core count for an ESU pCore license?
4. Where does in-guest tooling read managed-identity tokens from, with no stored secret?
5. Why must you avoid passing the SPN secret on the azcmagent connect command line?

Answers

1. Microsoft Entra ID (login.microsoftonline.com, pas.windows.net) and Azure Resource Manager (management.azure.com). Allow the AzureActiveDirectory and AzureResourceManager service tags or the agent can’t authenticate.
2. A remediation task. DeployIfNotExists/Modify act only on new/updated resources; run az policy remediation create against the assignment to bring the existing fleet in.
3. 16 physical cores. (vCore minimum is 8 per VM; Datacenter is only valid with pCore.)
4. The local Hybrid IMDS endpoint at http://localhost:40342, brokered by the agent’s system-assigned managed identity.
5. azcmagent can echo command-line arguments to logs in some failure paths, leaking the secret. Use a 0600 --config file and shred it after onboarding (or use certificate auth).

Glossary

• Azure Arc-enabled servers — Projection of an off-Azure machine into ARM as a Microsoft.HybridCompute/machines resource so Azure governance applies.
• Connected Machine agent (azcmagent) — The single package (CLI + HIMDS, GuestConfig, extension-manager services) that connects and manages an Arc server; outbound-only on 443.
• Hybrid Instance Metadata Service (HIMDS) — Local service at localhost:40342 that brokers managed-identity tokens and metadata in-guest.
• System-assigned managed identity — Per-machine, certificate-backed identity created at connect; lets in-guest scripts authenticate to Azure with no stored secret.
• Machine Configuration — In-box DSC-style engine that audits and optionally remediates in-guest state (registry, files, services, sysctl).
• Guest assignment — A Machine Configuration package bound to a machine (guestConfigurationAssignments), carrying the compliance status.
• assignmentType — Whether a guest assignment is Audit, ApplyAndMonitor, or ApplyAndAutoCorrect.
• Extended Security Updates (ESU) — Paid security patches for out-of-support Windows Server 2012/2012 R2, delivered off-Azure through Arc.
• ESU license — First-class ARM resource (HybridCompute/licenses) you provision and link to eligible machines.
• License profile — The machines/licenseProfiles/default child that links an ESU license to a machine.
• Connectivity mode — direct, proxy, or Private Link; shapes which endpoints go public vs private.
• Azure Arc Private Link Scope — HybridCompute/privateLinkScopes resource that routes the his/guestconfiguration data planes over a private endpoint.
• Onboarding service principal — A dedicated, RG-scoped Entra identity with Azure Connected Machine Onboarding used to connect machines non-interactively.
• Remediation task — The action that brings existing resources into compliance for DeployIfNotExists/Modify policies.
• Service tag — A named IP range (e.g. AzureActiveDirectory, AzureResourceManager) used in NSG/firewall rules to allow the always-public agent egress.

Next steps

• Azure Arc-Enabled Kubernetes: GitOps, Policy & Fleet Management — extend the same control-plane projection from servers to clusters.
• Azure Update Manager: Maintenance Configurations & Patch Orchestration with Arc — orchestrate patching (including ESU delivery) across the fleet you just onboarded.
• Defender for Servers: CWPP for Hybrid & Multicloud — add EDR and vulnerability assessment to your Arc machines.
• Azure Policy: Governance at Scale — deepen the policy assignments, initiatives, and remediation that drive Arc compliance.
• Enterprise-Scale Landing Zone: Management-Group Hierarchy Design — place your Arc estate inside a governed landing-zone hierarchy.