Configuring Azure Management Group Hierarchy Using Azure DevOps

Previously, I have published a 3-part blog series on deploying Azure Policy Definitions via Azure DevOps (Part 1, Part 2, Part 3). It covered one aspect of implementing Azure Governance using code and pipelines. There are at least 2 additional areas I haven’t covered:

1. Configuring Management Group hierarchy
2. Policy & Initiative assignments

In this post, I’ll cover how I managed to implement the management group hierarchy using Azure DevOps. I will cover policy & initiative assignment in a future blog post.

Problem Statement

Before I dive into the technical details, I’d like to firstly explain why is this required?

In an enterprise environment, subscriptions get created, renamed, disabled all the time. your cloud team may create Enterprise Agreement or Dev Test subscriptions from the EA accounts, some users may have sponsorship or MSDN subscriptions that they have enabled in the organisation’s Azure AD tenant and others may even have created free trial subscriptions also under the organisation’s tenant. For example, your tenant may look something like this:

When new subscriptions are joining your organisations tenants so frequently, you will need to make sure these subscriptions are automatically placed into appropriate management groups so you can apply appropriate policy and RBAC role assignments to them. It is not enough that you only manage the subscriptions that you know of. In the environment from the screenshot above, no one knew there are so many free trial and MSDN subscriptions in this particular tenant, until I enabled User Access Administrator role for myself(https://docs.microsoft.com/en-us/azure/role-based-access-control/elevate-access-global-admin).

So why do I care about these subscriptions? In the subscriptions under the management of the cloud team, we have many restrictions in place – for example, normal users cannot create vnets, public-facing storage accounts, etc. but these users can do whatever they want in their own MSDN or free trial subscriptions. users can also move resources between subscriptions (https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-move-resources). This can be a potential security risk, if you do nothing about these user-owned subscriptions.

Solution

Since everything should be driven by code and templates, Using Azure DevOps to implement the overall management group hierarchy makes perfect sense.

All the source code for my solution can be found at my GitHub repo: https://github.com/tyconsulting/Azure.ManagementGroup.Hierarchy.Config

In my solution, the pipeline performs 2 tasks:

1. Create the management group hierarchy for the tenant
2. Move subscriptions to appropriate management groups based on subscription names and types

The management group hierarchy and subscription placement rules are defined in a JSON definition file, a PowerShell script called configure-managementGroups.ps1 reads the definition file, firstly create the management group hierarchy, then scan through all the subscriptions in the tenant and place them based on the placement rules defined in the definition file.

Here’s a sample definition file:

{
  "tenantRootDisplayName": "Tao Yang Lab Root",
  "managementGroups":[
    {
      "name":"mg-mgmt-root",
      "displayName": "Management Root"
    },
    {
      "name":"mg-wl-root",
      "displayName": "Workload Root"
    },
    {
      "name":"mg-scf-root",
      "displayName": "Scaffolding Root"
    },
    {
      "name":"mg-scf-mgmt",
      "displayName": "Scaffolding Management",
      "parent": "mg-scf-root"
    },
    {
      "name":"mg-scf-wl",
      "displayName": "Scaffolding Workload",
      "parent": "mg-scf-root"
    },
    {
      "name":"mg-quarantine",
      "displayName": "Quarantine"
    }
  ],
  "subscriptionPlacements": [
      {
        "subNameRegex": "^the-big*",
        "subQuotaIdRegex": "^sponsored*",
        "managementGroup": "mg-wl-root"
      },
      {
        "subNameRegex": "^mvp*",
        "subQuotaIdRegex": "^msdn*",
        "managementGroup": "mg-mgmt-root"
      }
  ],
  "defaultManagementGroup": "mg-quarantine"
}

the definition file contains the following sections:

• Display name for the tenant root management group
• Management group hierachy (in the managementGroups section). this section defines the management groups to be created. each item contains the mandatory attributes of name and displayName. if the parent attribute is not specified, it will be placed under the tenant root MG.
• subscription placement rules (in the subscriptionPlacements section). Each rule contains 3 mandatory fields:
  • subNameRegex: the regular expression for the subscription name
  • subQuotaIdRegex: the regular expression for the subscription quota Id (more on this later)
  • managementGroup: the management group that the subscription is placed under when BOTH name and quota Id regex are matched.
• Default Management group: defines which management group should the subscription be placed under if it does not match any of the subscription placement rules.

Subscription Quota Id

Initially, I wanted to use the subscription Offer Id to identify the type of subscription (as shown below):

Then I learned that this offer Id is not exposed in any of the REST APIs (or at least I haven’t been able to find a way to query this field). Luckily, the list subscriptions ARM REST API exposed a similar attribute called quota Id. If you try to invoke this API from the docs site, you can quickly find out the corresponding quota Id for your subscriptions:

For example, I have few Azure Sponsorship subs, the quota Id for those subs is “Sponsored_2016-01-01”, and my MSDN sub is “MSDN_2014-09-01”. An EA sub is “EnterpriseAgreement_2014-09-01”, and an Enterprise MSDN Dev/Test sub is “MSDNDevTest_2014-09-01”.

Once you get the the subscription quota Id, you are able to build the regular expression for it. for example, a valid regex for enterprise sub can be “^EnterpriseAgreement_”, and for sponsorship sub: “^Sponsored_”.

Subscription placement rules

if your “management” subscriptions are created as EA subs and have a naming convention of “sub-mgmt-xxxx” and you wish to place these subs into a management group called “mg-mgmt-root”, the rule can be something like:

{
  "subNameRegex": "^sub-mgmt-*",
  "subQuotaIdRegex": "^EnterpriseAgreement_*",
  "managementGroup": "mg-mgmt-root"
}

Or if you want to place all users MSDN subscriptions into a management group called “mg-msdn”, the rule can be something like:

{
  "subNameRegex": "*",
  "subQuotaIdRegex": "^msdn_*",
  "managementGroup": "mg-msdn"
}

In order to be placed into the defined management group, the subscription must match both the name regex and quota Id regex. The script uses case insensitive match for the regex. so it doesn’t matter if you define your regex as “^msdn_” or “^MSDN_”.

Building the Pipelines

Before building the pipelines, there are some pre-requisites need to be taken care of first.

Pre-requisites

Azure AD App and Service Principal

You will need to create an Azure AD application with a service principal for each tenant that you are going to configure the management group hierarchy for. The service principal will need to have the owner role assigned at the tenant root management group level.

You may use my New-AADServivcePrincipal.ps1 script to create the service principal. For Example:

New-AADServicePrinicipal.ps1 –AADAppName AzDevOpsConnection –KeyType ‘Key’

Service Connection in Azure DevOps project

Once the service principal is created, you will need to create a service connection in the Azure DevOps project (for each tenant you are deploying this solution to):

• Scope level: ManagementGroup
• Management Group ID:

Build Pipeline

The build (CI) pipeline is defined in the pipelines/build-pipeline.yaml file in the GitHub repo. you can simply import it, it should work without any modifications.

It performs the following tasks:

1. installing required PowerShell modules from PowerShell Gallery (since I’m using Microsoft-hosted Azure DevOps agents here)
2. Validate the schema of all input files located in the “config-files” folder.
3. Publish test results
4. Publish pattern (copying artifacts to build staging directory for release pipelines)

Release Pipelines

Since subscriptions gets added to the tenant all the time, this script needs to run on a schedule (via Azure Pipelines). With the recent introduction of multi-stage YAML pipeline in Azure DevOps, we are able to combine build and releases in one YAML pipeline. However, because I am planning to schedule the release pipeline to run twice daily with the same build artifacts, there is no point creating a new build each time the pipeline runs. Therefore, I have separated the release pipeline from the build YAML pipeline, and used the classic pipeline instead, so I can only run the release pipeline on a schedule, using the same build artifacts.

I have enabled continuous deployment, and created two schedules:

For each stage in the release pipeline, I run 2 Azure PowerShell tasks:

1. The first task executes configure-managementGroups.ps1 with the –whatif switch to perform a dry run against the input file provided for the stage. It ensures there are no errors with the given input file before continuing to the next step.
2. If the first task completes successful, the second task performs the “real-run” to configure the hierarchy and placing subscriptions in management groups.

Step 1: What-if Dry run:

• Task Version: 4.* – make sure you choose 4 (or later) so the task supports the Az PowerShell module (instead of AzureRM)
• Script Path: path to the config-managementGroups.ps1 script, which is copied to the build staging folder by the build pipeline. use the “…” button to browse to the file.
• Script Arguments: –inputFile –silent –whatif
• ErrorActionPreference: Stop. So if the dry run fails, the pipeline will not move the next step
• Azure PowerShell Version: Latest installed version

Step 2: The “Real run”

This is pretty much the same as the first step, except the “-whatif” argument has been removed from the script arguments.

NOTE: When this step runs, the scripts will skip any existing management groups that are defined in the input file, and skip any subscriptions that are already placed in the correct management group. The script will DO NOTHING to any existing management groups that are not defined in the input file.

Conclusion

In this post, I have explained the solution that I have developed to implement Azure Management Group hierarchy in a very dynamic enterprise environment. Before implementing this solution, probably is better to firstly draw up your management group hierarchy design, figure out where in the hierarchy you are going to create RBAC assignments, custom policy / initiative definitions, and assigning policies / initiatives. If you have subscriptions created outside of your control, consider what kind of restrictions you wish to apply to these subs, and potentially advise your users move their non-EA subs to their own tenants.

In the future, it would be good if we can restrict our tenants to only allow certain types of subscriptions. But as far as I know, at the time of writing, I don’t believe we can configure this by ourselves.