Tag Archives: MimboloveOMS

PowerShell Module for OMS HTTP Data Collector API

Written by Tao Yang

Background

Earlier today, the OMS Product Group has released the OMS HTTP Data Collection API to public preview. If you haven’t read the announcement, you can read this blog post written by the PM of this feature, Evan Hissey first.

As a Cloud and Datacenter Management MVP, I’ve had private preview access to this feature for few months now, and I actually even developed a solution using this API in a customer engagement with my friend and fellow CDM MVP Alex Verkinderen (@AlexVerkinderen) just over a month ago. I was really impressed with the potential opportunities this feature may bring to us, I’ve been spamming Evan’s inbox asking him for the release date of this feature so I can blog about it and also present this in user group meetups.

Since most of us wouldn’t like having to deal with HTTP headers, bodies, authorizations and other overhead we have to put into our code in order to use this API, I have developed a PowerShell module to help us easily utilize this API.

Introducing OMSDataInjection PowerShell Module

This module was developed about 2 months ago, I was waiting for the API to become public so I can release this module. So now the wait is over, I can finally release it.

This module contains only one public function: New-OMSDataInjection. This function is well documented in a proper help file. you can access it via Get-Help New-OMSDataInjection –Full. I have added 2 examples in the help file too:

————————– EXAMPLE 1 ————————–

PS C:\>$PrimaryKey = Read-Host -Prompt ‘Enter the primary key’
$ObjProperties = @{
Computer = $env:COMPUTERNAME
Username = $env:USERNAME
Message  = ‘This is a test message injected by the OMSDataInjection module. Input data type: PSObject’
LogTime  = [Datetime]::UtcNow
}
$OMSDataObject = New-Object -TypeName PSObject -Property $ObjProperties
$InjectData = New-OMSDataInjection -OMSWorkSpaceId ‘8eb61d08-133c-401a-a45b-0e611194779f’ -PrimaryKey $PrimaryKey -LogType ‘OMSTestData’ -UTCTimeStampField ‘LogTime’ -OMSDataObject $OMSDataObject

Injecting data using a PS object by specifying the OMS workspace Id and primary key
————————– EXAMPLE 2 ————————–

PS C:\>$OMSConnection = Get-AutomationConnection ‘OMSConnection’
$OMSDataJSON = @”
{
“Username”:  “administrator”,
“Message”:  “This is a test message injected by the OMSDataInjection module. Input data type: JSON”,
“LogTime”:  “Tuesday, 28 June 2016 9:08:15 PM”,
“Computer”:  “SERVER01”
}
“@
$InjectData = New-OMSDataInjection -OMSConnection $OMSConnection -LogType ‘OMSTestData’ -UTCTimeStampField ‘LogTime’ -OMSDataJSON $OMSDataJSON

Injecting data using JSON formatted string by specifying the OMSWorkspace Azure Automation / SMA connection object (to be used in a runbook)

This PS module comes with the following features:

01. A Connection object for using this module in Azure Automation and SMA.

Once imported into your Azure Automation account (or SMA for the ‘old skool’ folks), you will be able to create connection objects that contains your OMS workspace Id, primary key and secondary key (optional):

image

And as shown in Example 2 listed above, in your runbook, you can retrieve this connection object and use it when calling the New-OMSDataInjection function.

02. Fall back to the secondary key if the primary key has failed

When the optional secondary key is specified, if the web request using the primary key fails, the module will fall back to the secondary key and try the web request again using the secondary key. This is to ensure your script / automation runbooks will not be interrupted when you are in the process of  following the best practice and cycling through your keys.

03. Supports two types of input: JSON and PSObject

As you can see from Evan’s post, this API is expecting a JSON object as the HTTP body which contains the data to be injected into OMS. When I started testing this API few months ago, my good friend and fellow MVP Stanislav Zhelyazkov (@StanZhelyazkov) suggested me instead of writing plain JSON format, it’s better to put everything into a PSObject then convert it to JSON in PowerShell so we don’t mess with the format and type of each field. I think it was a good idea, so I have coded the module to take either JSON format, or a PSObject that contains the data to be injected into OMS.

Sample Script  and Runbook

I’ve created a sample script and a runbook to help you get started. They are also included in the Github repository for this module (link at the bottom of this article):

Sample Script: Test-OMSDataInjection.ps1

Sample Runbook: Test-OMSDataInjectionRunbook

Exploring Data in OMS

Once the data is injected into OMS, if you are using a new data type,  it can take a while (few hours) for all the fields to be available in OMS.

i.e. the data injected by the sample script and Azure Automation runbook (executed on Azure):

image

all the fields that you have defined are stored as custom fields in your OMS workspace:

image

Please keep in mind, since the Custom Fields feature is still at the preview phase, there’s a limit of 100 custom fields per workspace at this stage (https://azure.microsoft.com/en-us/documentation/articles/log-analytics-custom-fields/), so please be mindful of this limitation when you are building your custom solutions using the HTTP Data Collector API.

Where to Download This Module?

I have published this module in PowerShell Gallery: https://www.powershellgallery.com/packages/OMSDataInjection, if you are using PowerShell version 5 and above, you can install it directly from it: Install-Module –Name OMSDataInjection –Repository PSGallery

You can also download it from it’s GitHub repo: https://github.com/tyconsulting/OMSDataInjection-PSModule/releases

Summary

In the past, we’ve had the OMS Custom View Designer that can help us visualising the data that we already have in OMS Log Analytics, what we were missing is a native way to inject data into OMS. Now with the release of this API, the gap has been filled. Like Evan mentioned in his blog post, by coupling this API with the OMS View Designer (and even throwing Power BI into the mix), you can develop some really fancy solutions.

On 21st of September (3 weeks from now), I will be presenting at the Melbourne Microsoft Cloud and Datacenter Meetup (https://www.meetup.com/Melbourne-Microsoft-Cloud-and-Datacenter-Meetup/events/233154212/), my topic is Developing Your OWN Custom OMS Solutions. I will doing live demos creating solutions using the HTTP Data Collector API as well as the Custom View Designer. If you are from Melbourne, I encourage you to attend. I am also planning to record this session and publish it on YouTube later.

Lastly, if you have any suggestions for this PowerShell module, please feel free to contact me!

OMS Network Performance Monitor Power BI Report

Written by Tao Yang

imageI’ve been playing with the OMS Network Performance Monitor (NPM) today. Earlier today, I’ve released an OpsMgr MP that contains tasks to configure MMA agent for NPM. You can find the post here: http://blog.tyang.org/2016/08/22/opsmgr-agent-task-to-configure-oms-network-performance-monitor-agents/

The other thing I wanted to do is to create a Power BI dashboard for the data collected by OMS NPM solution. The data collected by NPM can be retrieved using OMS search query “Type=NetworkMonitoring”.

To begin my experiment, I created a Power BI schedule in OMS using above mentioned query and waited a while for the data to populate in Power BI

image

I then used 2 custom visuals from the Power BI Custom Visual Gallery:

01. Force-Directed Graph

image

02. Timeline

image

and I created an interactive report that displays the network topology based on the NPM data:

image

In this report, I’m using a built-in slicer (top left) visual to filter source computers and the timeline visual (bottom) to filter time windows. The main section (top right) consists of a Force-Directed Graph visual, which is used to draw the network topology diagram.

I can choose one or more source computers from the slicer, and choose a time window from the timeline visual located at the bottom.

On the network topology (Force-Directed Graph visual), the arrow represents the direction of the traffic, thickness represents the median network latency (thicker = higher latency), and the link colour represents the network loss health state determined by the OMS NPM solution (LossHealthState).

I will now explain the steps I’ve taken to create this Power BI report:

01. Create a blank report based on the OMS NPM dataset (that you’ve created from the OMS portal earlier).

02. Create a Page Level Filter based on the SubType Field, and only select “NetworkPath”.

image

03. Add the Slicer visual to the top left and configure it as shown below:

image

image

04. Add the Force-Directed Graph (ForceGraph) to the main section of the report (top right), and configure it as shown below:

Fields tab:

  • Source – SourceNetworkNodeInterface
  • Target – DestinationNetworkNodeInterface
  • Weight – Average of MedianLatency
  • Link Type – LossHealthState

image

Format tab:

  • Data labels – On
  • Links
    • Arrow – On
    • Label – On
    • Color – By Link Type
    • Thickness – On
  • Nodes
    • Max name length – 15
  • Size – change to a value that suits you the best

image

05. Add a timeline visual to the bottom of the report, then drag the TimeGenerated Field from the dataset to the Time field:

image

As you can see, as long as you understand what each field means in the OMS data type that you are interested in, it’s really easy to create cool Power BI reports, as long as you are using appropriate visuals. This is all I have to share today, until next time, have fun in OMS and Power BI!

OpsMgr Agent Task to Configure OMS Network Performance Monitor Agents

Written by Tao Yang

OMS Network Performance Monitor (NPM) has made to public preview few weeks ago. Unlike other OMS solutions, for NPM, additional configuration is required on each agent that you wish to enrol to this solution. The detailed steps are documented in the solution documentation.

The product team has provided a PowerShell script to configure the MMA agents locally (link included in the documentation). In order to make the configuration process easier for the OpsMgr users, I have created a management pack that contains several agent tasks:

  • Enable OMS Network Performance Monitor
  • Disable OMS Network Performance Monitor
  • Get OMS Network Performance Monitor Agent Configuration

image

Note: Since this is an OpsMgr management pack, you can only use these tasks against agents that are enrolled to OMS via OpsMgr, or direct OMS agents that are also reporting to your OpsMgr management group.

These tasks are targeting the Health Service class, if you are also using my OpsMgr 2012 Self Maintenance MP, you will have a “Health Service” state view, and you will be able to access these tasks from the task pane of this view:

image

I can use the “Get OMS Network Performance Monitor Agent Configuration” task  to check if an agent has been configured for NPM.

i.e. Before an agent is configured, the task output shows it is not configured:

image

Then I can use the “Enable OMS Network Performance Monitor” task to enable NPM on this agent:

image

Once enabled, if I run the “Get OMS Network Performance Monitor Agent Configuration” task  again, the task output will show it’s enabled and also display the configured port number:

image

and shortly after, you will be able to see the newly configured node in OMS NPM solution:

image

If you want to remove the configuration, just simply run the “Disable OMS Network Performance Monitor” task:

image

You can download the sealed version of this MP HERE. I’ve also pushed the VSAE project for this MP to GitHub.

Visualising OMS Agent Heartbeat Data in Power BI

Written by Tao Yang

Introduction

Few days ago, the OMS product team has announced the OMS Agent Heartbeat capability. If you haven’t read about it, you can find the post here: https://blogs.technet.microsoft.com/msoms/2016/08/16/view-your-agent-health-in-oms-2/. In this post, Nini, the PM for the agent heartbeat feature explained how to create custom views within OMS portal to visualize the agent heartbeat data. Funny that I also started working on something similar around the same time, but instead of creating visual presentations within OMS, I did it Power BI. I managed to create couple of Power BI reports for the OMS agent heartbeat, using both native and custom Power BI Visuals:

01. Agent Locations Map Report:

Since the agent heartbeat data contains the geo location of the agent IP address, I’ve created this report to map the physical location the agent on an interactive map.

02. Agent Statistics Report:

This report has several parts, it contains the following parts:

  • A heat map based on the country where the agent is located (Agent Location by Country). The colour highlighting the country changes based on the agent count.
  • An interactive “fish tank” visual. In this visual, each fish represent an OMS agent. the size of the fish presents number of heartbeats generated by the agent. So, the older the agent (fish) is, the more heartbeat will be generated to the OMS workspace (fish tank), and the bigger the fish will become.
  • A Brick chart shows the percentage (this chart contains 100 tiles) of the agent by OS type (Linux vs Windows).
  • A tornado chart shows agent distribution by country. Agent OS type is also separated in different colours.
  • A Pie Chart shows agent distribution by management groups (SCOM attached vs direct attached vs Linux agents)
  • Agent version Donut chart that separates agent counts by agent version numbers (both Windows agents and Linux agents).

The fish visual is called “Enlighten Aquarium”, as you can see below, it’s an animated visual.

In this blog post, I will walk through the steps of creating these reports.

Instructions

Pre-Requisites

Before we create these reports, you need to make sure:

01. Power BI account

You will need to have a Power BI account (either a free or pro account) so OMS can inject data into your Power BI workspace.

02. Power BI preview feature is enabled in OMS

At the time of writing this post, the Power BI integration feature in OMS is still under public preview. Therefore if you haven’t done so, you will need to manually enable this feature first. To do so, go to the “Preview Features” tab in the OMS settings page, and enable “Power BI Integration”:

03. Connect your OMS workspace to your Power BI workspace.

Once the Power BI Integration feature is enabled, you need to connect OMS to Power BI. This is achieved by providing the Power BI account credential in the “Accounts” tab of the OMS settings page:

image

04. Setting up Power BI injection schedules

We need to inject the OMS agent heartbeat data to Power BI. We can just use a simple query: “Type=Heartbeat”, and set the schedule to run every 15 minutes:

image

05. Wait 15 – 30 minutes

You will have to wait a while before you can see the data in Power BI.

06. Download Power BI Custom visual

Since these reports use number of custom Power BI visuals, you will need to download them to your local computer first, and then import them into the reports when you start creating the reports. To download custom visuals, go to the Power BI Visuals Gallery (https://app.powerbi.com/visuals/) and download the following visuals:

  • Brick Chart
  • Hierarchy Slicer
  • Donut Chart GMO
  • Timeline
  • Tornado Chart
  • Enlighten Aquarium

Create Reports

To start creating the report, firstly logon to Power BI using the account you’ve used to make the connection in OMS, and then find the Dataset you have specified. in this post, I’ve created a dataset called “OMS – Agent Heartbeat”. By clicking on the dataset, you will be presented to an empty report:

image

You will then need to import the custom visuals – by clicking on the “…” icon under Visualizations, and select “Import a custom visual”

image

You can only import one at a time, so please repeat this process and import all the custom visuals I have listed above.

Creating Agent Location Report

For the Agent location report, we will add 3 visuals:

  • Hierarchy Slicer – for filtering IP addresses and computer names
  • Map – for pinpointing the agent location
  • Timeline – for filtering the time windows

image

Agent Filter (Hierarchy Slicer)

image

Add a Hierarchy slicer and place on the left side of the page, then drag the ComputerIP and Computer fields from to the “Fields” section, please make sure you place ComputerIP on top of Computer:

image

it’s also a good idea to turn off single selection for the hierarchy slicer so you can select multiple items:

image

Agent Location Map

image

Add the Map visual to the report, configure it as listed below:

  • Location – RemoteIPCountry
  • Legend – Computer
  • Latitude – Average of Remote IPLatitude
  • Longitude – Average of RemoteIPLongitude
  • Size – Count of Computer (Distinct)

image

Note: since the latitude and longitude shouldn’t change between different records for the same computer as long as the IP doesn’t change, so it doesn’t matter if you use average, or maximum or minimum, the result of each calculation should be the same.

Time Slicer (Timeline)

image

Add a timeline slicer to the bottom of the report page, configure it to use the TimeGenerated field:

image

To same some space on the report page, you may also turn off the labels for the timeline slicer:

image

Lastly, add a text box on the top of the report page, give it a title, also if you want to, assign each visual a title by highlighting the visual, then click on Format icon, and update the title field:

image

To use this report, you can make your selections in the hierarchy slicer and the timeline slicer. The map will be automatically updated.

Create Agent Statistic Report

For the second report, you can create a new page of the existing report, or create a brand new report based on the same dataset. We will use the following visuals in this report:

  • Filled Map
  • Aquarium
  • Brick Chart
  • Tornado Chart
  • Pie Chart
  • DonutChartGMO

image

Agent Location By Country (Filled Map)

image

Configure the Filled Map visual as shown below:

image

OMS Agent By Heartbeat Count (Aquarium)

image

Configure the Aquarium visual as shown below:

image

Agent OS Type (Brick Chart)

image

Configure the Brick Chart visual as shown below:

image

Agent Distribution By Country (Tornado Chart)

image

Configure the Tornado Chart as shown below:

image

Agent Distribution By Management Groups (Pie Chart)

image

Configure the Pie Chart as shown below:

image

Agent Version (DonutChartGMO)

image

Configure the DonutChartGMO visual as shown below:

image

and change the Primary Measure under Legend to “Value” / “Percentage” / “Both”, whichever you prefer:

image

Most of the visuals used by this report are interactive. i.e. if I click on a section in the Agent Version DonutChartGMO visual, other visuals will be automatically updated to reflect the selection I made in the DonutChartGMO visual.

Once you’ve configured all the visuals, please make sure you save your report.

Conclusion

There are many things you can do with the Power BI reports you’ve just created. i.e. you can share it with other people, ping individual visuals or entire report to a dashboard, or create an Iframe link and embed the report to 3rd party systems that support IFrame (i.e. SharePoint sites). We are not going to get into details of how to consume these reports today.

Please note that during my testing, the RemoteIPLatitude and RemoteIPLongitude data from the heartbeat events are not very accurate for the computers in my lab. I’m based in Melbourne, Australia but the map coordinates pinged to a location in Sydney, which is over 1000km away from me.

Please also be aware that for SCOM attached agents, each time when the agent sends heartbeat, it will send 2 heartbeats via different channels. This behaviour is by design – my good friend and fellow CDM MVP Stanislav Zhelyazkov(@StanZhelyazkov) has explained this in his blog post: https://cloudadministrator.wordpress.com/2016/08/17/double-heartbeat-events-in-oms-log-analytics/

This is all I have to share for today. until next time, have fun with OMS and Power BI!

OMS Near Real Time Performance Data Aggregation Removed

Written by Tao Yang

Few weeks ago, the OMS product team has made a very nice change for the Near Real Time (NRT) Performance data – the data aggregation has been removed! I’ve been waiting for the official announcement before posting this on my blog. Now Leyla from the OMS team has finally broke the silence and made this public: Raw searchable performance metrics in OMS.

I’m really excited about this update. Before this change, we were only able to search 30-minute aggregated data via Log Search. this behaviour brings some limitations to us:

  • It’s difficult to calculate average values based on other intervals (i.e. 5-minute or 10-minute)
  • Performance based Alert rules can be really outdated – this is because the search result is based on the aggregated value over the last 30 minutes. In critical environment, this can be a bit too late!

By removing the data aggregation and making the raw data searchable (and living a longer life), the limitations listed above are resolved.

Another advantage this update brings is, it greatly simplified the process of authoring your own OpsMgr performance collection rules for OMS NRT Perf data. Before this change, the NRT perf rules come in pairs – each perf counter you want to collect must have 2 rules (with the identical data source module configurations). One rule is for collecting raw data and another is to collect the 30-minute aggregated data. This has been discussed in great details in Chapter 11 of our Inside Microsoft Operations Management Suite book (TechNet, Amazon). Now, we no longer need to write 2 rules for each perf counter. We only need to write one rule – for the raw perf data.

The sample OpsMgr management pack below collects the “Log Cache Hit Ratio” counter for SQL Databases. It is targeting the Microsoft.SQLServer.Database class, which is the seedclass for pre-SQL 2014 databases (2005, 2008 and 2012):

As you can see from the above sample MP, the rule that collects aggregated data is no longer required.

image

So if you have written some rules collecting NRT perf data for OMS in the past, you may want to revisit what you’ve done in the past and remove the aggreated data collection rules.

ConfigMgr OMS Connector

Written by Tao Yang

Earlier this week, Microsoft has release a new feature  in System Center Configuration Manager 1606 called OMS Connector:

image

As we all know, OMS supports computer groups. We can either manually create computer groups in OMS using OMS search queries, or import AD and WSUS groups. With the ConfigMgr OMS Connector, we can now import ConfigMgr device collections into OMS as computer groups.

Instead of using the OMS workspace ID and keys to access OMS, the ConfigMgr OMS connector requires an Azure AD Application and Service Principal. My friend and fellow Cloud and Data Center Management MVP Steve Beaumont has blogged his setup experience few days ago. You can read Steve’s post here: http://www.poweronplatforms.com/configmgr-1606-oms-connector/.  As you can see from Steve’s post, provisioning the Azure AD application for the connector can be pretty complex if you are doing it manually – it contains too many steps and you have to use both the old Azure portal (https://manage.windowsazure.com) and the new Azure Portal (https://portal.azure.com).

To simplify the process, I have created a PowerShell script to create the Azure AD application for the ConfigMgr OMS Connector. The script is located in my GitHub repository: https://github.com/tyconsulting/BlogPosts/tree/master/OMS

In order to run this script, you will need the following:

  • The latest version of the AzureRM.Profile and AzureRM.Resources PowerShell module
  • An Azure subscription admin account from the Azure Active Directory that your Azure Subscription is associated to (the UPN must match the AAD directory name)

When you launch the script, you will firstly be prompted to login to Azure:

image

Once you have logged in, you will be prompted to select the Azure Subscription and then specify a display name for the Azure AD application. If you don’t assign a name, the script will try to create the Azure AD application under the name “ConfigMgr-OMS-Connector”:

SNAGHTMLc560723

This script creates the AAD application and assign it Contributor role to your subscription:

image

At the end of the script, you will see the 3 pieces of information you need to create the OMS connector:

  • Tenant
  • Client ID
  • Client Secret Key

You can simply copy and paste these to the OMS connector configuration.

Once you have configured the connector in ConfigMgr and enabled SCCM as a group source, you will soon start seeing the collection memberships being populated in OMS. You can search them in OMS using a search query such as “Type=ComputerGroup GroupSource=SCCM”:

image

Based on what I see, the connector runs every 6 hours and any membership additions or deletions will be updated when the connector runs.

i.e. If I search for a particular collection based on the last 6 hours, I can see this particular collection has 9 members:

image

During my testing, I deleted 2 computers from this collection few days ago. If I specify a custom range targeting a 6-hour time window from few days ago, I can see this collection had 11 members back then:

image

This could be useful sometimes when you need to track down if certain computers have been placed into a collection in the past.

This is all I have to share today. Until next time, enjoy OMS Smile.

Scoping OMS Performance Data in Power BI

Written by Tao Yang

when working on a dashboard or a portal, sometimes it is good that the portal is more interactive. I often found it’s more useful then just a static widget. Since I come from the monitoring back ground, I’ll use performance data as an example.

In the good old SCOM, we have this awesome 3rd party web portal called Squared Up, which allows you to choose the time frame for the perf graph:

image

and you can also select the time frame by highlighting a section from the graph itself:

image

In OMS, when we are playing with the Near Real-Time (NRT) Performance data (Type=Perf), we also have the options to specify the time frame of our choice:

image

Additionally, if we have chosen a time scope that is 6 hours or less, we are able to see the raw NRT perf data coming in every few seconds (in light blue colour):

image

Both Squared Up (for SCOM) and OMS portal provides very interactive ways to consume the perf data.

As we all know, OMS has the ability to send collected data to Power BI, therefore we are also able to create Power BI reports that contains performance data injected by OMS. i.e.:

image

As you can see, with the Power BI Line Chart visual, we can even add a trend line (the black dotted line), which is very nice in my opinion. However, by using native visuals, there are few limitations with displaying performance data in Power BI:

  • The time frame cannot be easily scoped
  • The computer and performance counters cannot be easily scoped

What I mean is, you can absolutely create a filters on either visual level, or page level or even the report level to create desired scopes – just like what I did in the example above:

image

But these filters are rather static. You won’t be able to alter them once you’ve saved the report. Obviously, as the report creator, you don’t really want to multiple almost identical visuals for different counters for different computers. In my opinion, reports like these become less interactive and user friendly because they are too static.

So, how do we make these Power BI reports more interactive? there are few options:

1. Use a Slicer to filter the computers OR the counters

In Power BI, you can add a slicer to your page. Slicers makes the report more interactive. users can choose one or more items from the slicer and other visuals on the page will be updated automatically based on users selection.

image

In the above example, I’ve used page level filter to only display the ‘Availability MBytes’ counter, and users can use the slicer to choose the computers they are interested in.

This solution is easy to implement, it may satisfy the requirements if you are only interested in a specific counter from a long term trend point of view – since we are not filtering the time windows, it will display the entire period that is available in Power BI.

2. Use the Custom Visual ‘Hierarchy Slicer’ to filter the computers AND the counters

For Power BI, you can download custom visuals from https://app.powerbi.com/visuals/?WT.mc_id=Blog_CustomVisuals and then import into your reports.

One of the custom visual you can download is called Hierarchy Slicer:

image

As the name suggests, comparing to the original built-in slicer, this visual allows you to build a hierarchy for your slicers:

image

As you can see, I’ve added Computer name as the top level filter in the hierarchy slicer, followed by the counter name as the second level in the slicer. As the result, I don’t have to use the filters for this page. Users can simply click on a counter (2nd level)  to view the graph for the counter on that specific computer, or select a computer (1st level) to see all the perf data for that particular computer. Obviously, you can make the counter name as the top of the hierarchy and place the computer name as the second level if that suits your needs better.

Note: As per introduction video for this visual, you can enable multi-select by configuring the visual and turn off the ‘Single Select’ option:

image

However, based on my experience, this option is only available when you are using Power BI Desktop. It is not available in Power BI Online.

image

Therefore we won’t be able to use multi-select for the OMS injected data because we cannot use Power BI Desktop with OMS data.

3. Use the Brush Chart custom visual to scope the time frame

Another cool custom visual is called Brush Chart, it is also called ‘Advanced Time Slicer’ on the download page:

image

I am using this together with the hierarchy slicer, so I can scope both computers and counters, as well as the perf data time window.

image

As you can see, there are 2 graphs on this visual. I can use mouse (or other pointing devices) to select a time window from the bottom graph, and the top graph will be automatically zoomed into the selected time period.

4. Use the Time Brush Custom Visual to scope the time frame

The Time Brush custom visual is very similar to the Brush Chart (aka Advanced Time Slicer).

image

It cannot be used by itself, it acts as the control for other visuals. in the example below, I’m using it together with the Line Chart visual, as well as the hierarchy slicer:

image

As you can see, when I select a period from the Time Brush visual, the line chart got updated automatically.

5. use other custom visuals

There are a lot of other custom visuals that you can download. for example, there’s another time slicer called TimeLine that allows you specify a precise,  specific time frame.

image

Conclusion

By using the combination of various slicers, we can produce more interactive and user friendly reports in Power BI. In the examples listed above, I can quickly produce a single report for ALL the OMS performance data, and users can simply choose the computer, counter and the time frame from the report itself. There is no need to create separate reports for different counters or computers.

I hope you find these tips useful, and have fun with OMS and Power BI!

HybridWorkerToolkit PowerShell Module Updated to Version 1.0.3

Written by Tao Yang

Few days ago, I published a PowerShell Module to be used on Azure Automation Hybrid Workers called HybridWorkerToolkit. You can find my blog article HERE.

Yesterday, my good friend and fellow CDM MVP Daniele Grandini (@DanieleGrandini) gave me some feedback, so I’ve updated the module again and incorporated Daniele’s suggestions.

This is the list of updates in this release:

  • A new array parameter for New-HybridWorkerEventEntry called “-AdditionalParameters”. This parameter allows users to insert an array of additional parameters to be added in the event data:

SNAGHTMLb6e7547

  • A new Boolean parameter for New-HybridWorkerEventEntry called “-LogMinimum”. This is an optional parameter with the default value of $false. When this parameter is set to true, other than the user specified messages and additional parameters, only the Azure Automation Job Id will be logged as event data:

image

As we all know, we pay for the amount of data gets injected into our OMS workspace, this parameter allows you to minimise the size of your events (thus saves money on your OMS spending).

I have published this new release to both GitHub and PowerShell Gallery.

New PowerShell Module HybridWorkerToolkit

Written by Tao Yang

HybridWorkerToolkit23/04/2016 Update: released version 1.0.3 to GitHub and PowerShell gallery. New additions documented in this blog post.

21/04/2016 Update: updated GitHub and PowerShell gallery and released version 1.0.2 with minor bug fix and updated help file.

Introduction

Over the last few days, I have been working on a PowerShell module for Azure Automation Hybrid Workers. I named this module HybridWorkerToolkit.

This module is designed to run within either a PowerShell runbook or a PowerShell workflow runbook on Azure Automation Hybrid Workers. It provides few functions that can be called within the runbook. These activities can assist gathering information about Hybrid Workers and the runbook runtime environment. It also provides a function to log structured events to the Hybrid Workers Windows Event Logs.

My good friend and fellow MVP Pete Zerger posted a method he developed to use Windows event logs and OMS as a centralised logging solution for Azure Automation runbooks when executed on Hybrid Workers. Pete was using the PowerShell cmdlet Write-EventLog to log runbook related activities to Windows event log and then these events will be picked up by OMS. Log Analytics. This is a very innovative way of using Windows event logs and OMS. However, the event log entries written by Write-EventLog are not structured are lacking basic information about your environment and the job runtime.  Couple of weeks ago, another friend of mine, Mr. Kevin Holman from Microsoft also published a PS script that he used to write to Windows event logs with additional parameters.

So I combined Pete’s idea with Kevin’s script, as well as some code I’ve written in the past for Hybrid Workers, and developed this module.

Why do we want to use Windows Event logs combined with OMS for logging runbook activities on Hybrid workers? As Pete explained on this post, it provides a centralised solution where you can query and retrieve these activity logs for all your runbooks from a single location. Additionally, based on my experience (and also confirmed with few other friends), is that when you use Write-Verbose or Write-Output in your runbook and enabled verbose logging, the runbook execution time can increase significantly, especially when loading a module with a lot of activities. Based on my own experience, I’ve seen a runbook that would normally takes a minute or two to run with verbose logging turned off ended up ran over half an hour after I enabled verbose logging. This is another reason I’ve developed this module so it gives you an alternative option to log verbose, error, process and output messages.

Functions

This module provides the following 3 functions:

  • Get-HybridWorkerConfiguration
  • Get-HybridWorkerJobRuntimeInfo
  • New-HybridWorkerRunbookLogEntry

Note: Although the job runtime are different between PowerShell runbooks and PowerShell Workflow runbooks, I have spent a lot of time together with Pete making sure we can use these activities exactly the same ways between PowerShell and PowerShell workflow runbooks.

Get-HybridWorkerConfiguration

This function can be used to get the Hybrid Worker and Microsoft Monitoring Agent configuration. A hash table is returned the following configuration properties retrieved from Hybrid Worker and MMA agent:

  • Hybrid Worker Group name
  • Automation Account Id
  • Machine Id
  • Computer Name
  • MMA install root
  • PowerShell version
  • Hybrid Worker version
  • System-wide Proxy server address
  • MMA version
  • MMA Proxy URL
  • MMA Proxy user name
  • MMA connected OMS workspace Id

Get-HybridWorkerJobRuntimeInfo

This function retrieves the following information about the Azure Automation runbook and the job run time. They are returned in a hashtable:

  • Runbook job ID
  • Sandbox Id
  • Process Id
  • Automation Asset End Point
  • PSModulePath environment variable
  • Current User name
  • Log Activity Trace
  • Current Working Directory
  • Runbook type
  • Runbook name
  • Azure Automation account name
  • Azure Resource Group name
  • Azure subscription Id
  • Time taken to start runbook in seconds

New-HybridWorkerRunbookLogEntry

This function can be used to log event log entries. By default, other than the event message itself, the following information is also logged as part of the event (placed under the <EventData> XML tag:

  • Azure Automation Account Name
  • Hybrid Worker Group Name
  • Azure Automation Account Resource Group Name
  • Azure Subscription Id
  • Azure Automation Job Id
  • Sandbox Id
  • Process Id
  • Current Working Directory ($PWD)
  • Runbook Type
  • Runbook Name
  • Time Taken To Start Running in Seconds

This function also has an optional Boolean parameter called ‘-LogHybridWorkerConfig’ When this parameter is set to $true, the event created by this function will also contain the following information about the Hybrid Worker and MMA:

  • Hybrid Worker Version
  • Microsoft Monitoring Agent Version
  • Microsoft Monitoring Agent Install Path
  • Microsoft Monitoring Agent Proxy URL
  • Hybrid Worker server System-wide Proxy server address
  • Microsoft OMS Workspace ID

Sample Runbooks

Sample PowerShell Runbook:

Sample PowerShell Workflow Runbook

As you can see, the way to call these functions between PowerShell and PowerShell Workflow runbooks are exactly the same.

Hybrid Worker Configuration output:

SNAGHTML40e35ad

Hybrid Worker Job Runtime Info output:

SNAGHTML40f4d28

Event generated (with basic information / without setting –LogHybridWorkerConfig to $true):

SNAGHTML4159a60[4]

Event generated (whensetting –LogHybridWorkerConfig to $true):

SNAGHTML4150515

Consuming collected events in OMS

Once you have collected these events in OMS, you can use search queries to find them, and you can also create OMS alerts to notify you using your preferred methods.

Searching Events in OMS

i.e. I can use this query to get all events logged by a particular runbook:

Type=Event “RunbookName: Test-HybridWorkerOutput-PSW”

image

or use this query to get all events for a particular job:

Type=Event “JobId: 73A3827D-73F8-4ECC-9DE1-B9340FB90744”

image

OMS Alerts

i.e. if I want to create an OMS alert for any Error events logged by New-HybridWorkerRunbookLogEntry, I can use a query like this one:

Type=Event Source=AzureAutomation?Job* EventLevelName=Error

image

image

Download / Deploy this module

I have published this module on Github as well as PowerShell Gallery:

GitHub Repository: https://github.com/tyconsulting/HybridWorkerToolkit

PowerShell Gallery:  http://www.powershellgallery.com/packages/HybridWorkerToolkit/1.0.3

Credit

I’d like to thank Pete and Kevin for the ideas in the first place, also I’d like to thank Pete, Jakob Svendsen, Daniele Grandini and Kieran Jacobsen for the testing and feedback!

My Global Azure Boot Camp 2016 Presentation Recording

Written by Tao Yang

Last Saturday, I presented the topic of “What’s New in OMS” at the Global Azure Boot Camp 2016 Melbourne event. I have recorded it using Camtasia and uploaded to YouTube. You can watch it here:

You can also download the slide deck from HERE. When I was editing the recording, I noticed that I might not placed the lapel microphone properly as it was rubbing my collar and you will notice the some noise in the video. Oh well, something to improve next time.