Monthly Archives: March 2017
Over the last few months, Stan, Pete, Anders and I have been very busy with writing the version 2 of the Inside Microsoft Operations Management Suite book. Although we still have few more chapters to finish, we have decided to release 3 preview chapters now.
The first preview chapter was released yesterday. It was Chapter 6: Extending OMS Using Log Search (http://insidethecloudos.azurewebsites.net/early-chapter-preview-of-inside-oms-version-2/). This chapter was written by myself, and reviewed by my MVP buddy Kevin Greene (@kgreeneit) and Pete himself. This chapter has covered several OMS functionalities that are based on Log search:
- Saved Searches
- OMS Computer Groups
- Custom Fields
- Custom Logs
- Power BI integration
Since there aren’t many documentations out there on for OMS Power BI integration feature, I have specifically spent a lot of time on the Power BI integration feature, it makes up about half of this 73-page chapter.
I must say that Kevin has done a fantastic job reviewing this chapter. Kevin’s review was really thorough, when I got Kevin’s feedback, it literally blew my mind! So special thanks to Kevin and Pete for their effort reviewing this chapter.
Lastly, the next 2 preview chapters will be released shortly. Please keep eye on Pete’s blog (http://insidethecloudos.azurewebsites.net).
Over the last few weeks, it occurred to me twice that I had to change my original design of the automation solutions I was working on because of the limitations of Azure Automation Azure Runbook Workers. Last month, my fellow CDM MVP Michael Rueefli has published an article and explained Why deploying Hybrid Runbook Workers on Azure makes sense. In Michael’s article, he listed some infrastructural differences between Azure runbook workers and the Hybrid runbook workers. However, the issues that I faced that made me to change my design were caused by the functional limitations in Azure runbook workers. Therefore I thought I’d post a supplement post to document my findings. Since these limitations are not documented in Microsoft’s official documentation site, please DO NOT assume this is the complete list, and it is still valid by the time you read it. I will try my best to keep this post up-to-date over the time. So, there are the limitations I found:
01. Windows Event Log Service is missing
Few months ago, I wrote a runbook that reads event log export .evt files and inject the records to OMS (http://blog.tyang.org/2016/12/05/injecting-event-log-export-from-evtx-files-to-oms-log-analytics/). This runbook uses a .NET class System.Diagnostics.Eventing.Reader.EventLogQuery to read events from the evt files. Few days ago, when I tried to implement a version of this runbook for a solution that I was initially planned to use Azure runbook workers, the runbook job failed when configured to run on Azure and I got a “RPC Server is unavailable” error.
After some troubleshooting, I found the cause of this error. The .NET class EventLogQuery relies on the Windows Event Log service to read the evt files, but in the Azure runbook worker’s sandbox environment, Windows Event Log service does not exist. Therefore, I have no choice but changing the design the having this runbook to be executed on the Hybrid runbook worker.
02. Unable to Map Network Drives
In a runbook, I have a code block that maps a network drive to an Azure storage File Share and read some files. When ran it on the Azure runbook worker, I found it was not possible to map the network drive. Luckily I could use the Azure.Storage PowerShell module to access the files in the Azure File Share, so I had to update the runbook to accommodate this limitation.
03. Disk Size Limitation
In a runbook, I needed to extract a 2GB zip file. It failed to run on Azure runbook worker because of the insufficient disk space and I couldn’t even copy the 2GB file to the Azure runbook worker. I attempted to find the size of the system drive on the Azure runbook workers by using Get-PSDrive cmdlet but it did not return the disk size.
04. Unable to use Service Principals and Credentials to Login to Azure
In a solution that I was working on, I designed to use Service Principals (Azure AD applications) and credentials to login to Azure. This method worked perfectly when the runbook was executed on the Hybrid runbook worker, but does not work when running on Azure. After some researching, I found someone had the same issue and posted on Stack Overflow: http://stackoverflow.com/questions/37619377/login-azurermaccount-credential-immediately-expiring. Basically, using credentials with service principals is not supported when the runbook is executed on Azure runbook workers.
Based on my experiences so far, I’ve come to a conclusion that I should not automatically assume everything is going to work on Azure runbook workers when designing my solution. In future, I will make sure that I’ll test early and every step along the way if I am planning to have the runbook executed on Azure runbook workers.
If you have experienced limitations that are not listed here, please let me know and I’m happy to add them onto this post.
Over the past decade, I have used several password management applications such as Password Safe, KeePass and LastPass. Out of these products, only LastPass is cloud based. I have been hesitate to use LastPass over the last few years and stayed with KeePass because of the LastPass data breach back in 2015. Few months ago, my friend Alex Verkinderen finally convinced me to start using LastPass again. But this time, in order to be more secure and being able to use Multi-Factor Authentication (MFA), I have purchased a premium account and also purchased a YubiKey Neo for MFA. I understand not everyone is willing to spend money on password repository solutions (in my case, USD $12 per year for the LastPass Premium account and USD $50 + shipping for a Yubikey Neo from Amazon). Also, based on my personal experience, there are still many organisations that don’t have a centralised password repositories. Many engineers and consultants I have met still store passwords in clear text.
On the other hand, Azure Key Vault has drawn a lot of attention since it was released and it is become really popular. I have certainly used it a lot over the last few months and managed to integrate it with many solutions that I have built.
AzureKeyVaultPasswordRepo PowerShell Module
I spent few hours last night and today, developed a PowerShell CLI menu based app based on few existing scripts I wrote in the past. This app allows you to create, manage Azure Key Vault and use it as your personal (or team’s) password repository. In order to simplify the process of deploying and using this app, I wrapped it in a PowerShell module. I named this module AzureKeyVaultPasswordRepo and it is now available on both PowerShell Gallery and GitHub:
PowerShell Gallery: https://www.powershellgallery.com/packages/AzureKeyVaultPasswordRepo/
If you are running PowerShell version 5 and later, you can install this module using an one-liner:
Once it is installed, you can launch the app either using the full name Invoke-AzureKeyVaultPasswordRepository, or use one of the 2 shorter aliases (ipr and Start-PasswordRepo).
This module requires AzureRm.Profile, AzureRm.Resources and AzureRm.KeyVault modules, which you can also find from the PowerShell Gallery.
When it is launched, it will detect if you are currently Signed in to Azure and ask you if you want to keep using the same account if you are currently signed in.
you have the option to keep using the current account or sign in to Azure using another account.
Then the app will prompt you to use the current Azure subscription that’s set in the context, or select another subscription from the list.
When running it for the first time, you will need to create a new Key Vault from the menu. You can choose an existing resource group, or create a new resource group in your azure region of your choice
Once the key vault is created, you will need to assign full access to an Azure AD account. This is done by searching Azure AD using a search string and select an user account from the search result list.
Once the permission is assigned, everything is ready to go. you will be presented with the main menu:
Note: It is by design that this app does not use any existing key vaults that you may already have in your subscription. You have to create a new one. Any existing key vaults that are not created by this app will not appear on the list for you to choose.
Creating Profile to store settings
In order to make you access this key vault as fast as possible in the future, the first thing I’d suggest you to do is to select option 4 and save the Azure subscription Id and Key Vault name in your profile. this profile is stored in Windows Registry under HKEY_CURRENT_USER\SOFTWARE\TYConsulting\AzureKeyVaultPasswordRepo\Profiles\<your Azure account name>.
Once the profile is saved, when you launch this app next time, it will automatically use the Azure subscription and the Key Vault that’s stored in the profile.
From the main menu, you have the option to:
1. Create new credential (user name and password). you also have the option to generate random password by not entering a password. if you choose to use this app to generate a random password, the password will be copied to the computer’s clipboard once the credential is created (so you can use Ctrl-V to paste it to wherever you need to).
List, Retrieve, Update and Delete Credentials
You can use Option 2 to list, retrieve, update and delete existing credentials. When option 2 is selected, the app will list all credentials stored in the key vault, and from there, you can choose the credential from the list that you are interested in. Once the credential is selected, you have the option to:
- Copy user name to clipboard
- Copy password to clipboard
- Update credential (username / password)
- Delete Credential
Instead of selecting a credential to manage from the list, you can also search credentials (based on credential name) using option 3.
Save / Delete Profile
As shown previously, for faster access, you can use option 4 to store the Azure subscription Id and the Key Vault Name in the registry. if you decide to delete the profile (i.e. when you decide to use another subscription or key vault), you can use option 5 to delete the existing profile from registry.
Managing Key Vault Access
You can use option 6 to grant full access to the key vault to other Azure AD accounts, or use option 7 to remove access.
Personally I’m pretty happy to see what I have produced during such a short period of time (only few hours based on some existing scripts I wrote few weeks ago). I think this would fill a gap for people and organisations that do not have a commercial password management solution.
Azure Key Vault is a very in-expensive solution, and by using an Azure offering, you automatically inherit the MFA solutions that you have configured for Azure / Azure AD. i.e. I’m not using Azure AD premium for my lab but for my Microsoft (@outlook.com) account, I have enabled MFA using the Microsoft Authenticator app. Therefore in order to access the Key Vault using this module, I will need to use MFA during the sign in process.
I’ve only spent few hours on this PowerShell module, there are still room for improvement. So consider this as a MVP (Minimum Viable Products). I think the following additions would be beneficial in future releases (if I decide to have develop further):
- A GUI interface
- Support additional types of sensitive information, not just username and passwords
- Support Service Principal (Azure AD Applications)
- Support different levels of access (currently everyone has full access)
Lastly, please give it a try, and I’d like to hear back from the community. If you are interested to learn how to interact with Key Vault using PowerShell, feel free to read the source code of this module. if you have any questions or suggestions, please feel free to contact me!