Session Recording for My Presentation in Microsoft MVP Community Camp Melbourne Event

Written by Tao Yang

image

Last Friday, I presented in the Melbourne MVP Community Camp day, on the topic of “Automating SCOM Tasks Using SMA”.

I have uploaded the session recording to YouTube. You can either watch it here:

If you’d like to watch it in full screen, please go to: https://www.youtube.com/watch?v=QW99bVFKg80

 

Or on YouTube: https://www.youtube.com/watch?v=QW99bVFKg80

You can also download the presentation deck from HERE.

And Here’s the sample script I used in my presentation when I explained how to connect to SCOM management group via SDK:

Overall, I think I could have done better – as I wasn’t in the best shape that day. I have been sick for the last 3 weeks (dry cough passed on to me from my daughter). The night before the presentation, I was coughing none-stop and couldn’t go to sleep. I then got up, looked up the Internet and someone suggested that sleeping upright might help. I then ended up slept on the couch for 2.5 hours before got up and drove to Microsoft’s office. So I was really exhausted even before I got on stage. Secondly, the USB external Microphone didn’t work on my Surface, so the sound was recorded from the internal mic – not the best quality for sure.

Anyways, for those who’s watching the recording online, I’m really interested in hearing back from you if you have any suggestions or feedbacks in regards to the session itself, or the OpsMgrExtended module that I’m about to release. So, please feel free to drop me an email if you like Smile.

Microsoft MVP Community Camp 2015 and My Session for SMA Integration Module: OpsMgrExtended

Written by Tao Yang

141215_comcamp2015_Melbourne_01

On next Friday (30th Jan, 2015), I will be speaking at the Microsoft MVP Community Camp Day in Melbourne. I am pretty excited about this event as this is going to be my first presentation since I have become a System Center MVP in July 2014.

My session is titled “Automating SCOM tasks using SMA”. Although this name sounds a little bit boring, let me assure you, it won’t be boring at all! The stuff I’m going to demonstrate is something I’ve been working on during my spare time over the last 6 month, and so far I’ve already written over 6,000 lines of PowerShell code. Basically, I have created a module called “OpsMgrExtended”. this module can be used as a SMA Integration Module as well as a standalone PoewrShell module. It directly interact with OpsMgr SDKs, and can be used by SMA runbooks or PowerShell scripts to perform some advanced tasks in OpsMgr such as configuring management groups, creating rules, monitors, groups, overrides, etc.

If you have heard or used my OpsMgr Self Maintenance MP, you’d know that I have already automated many maintenance / administrative tasks in this MP, using nothing but OpsMgr itself. In this presentation, I will not be showing you anything that’s already been done by the Self Maintenance MP. I will heavily focus on automating management pack authoring tasks.

To date, I haven’t really discussed this piece of work in details with anyone other than few SCOM focused MVPs (and my wife of course). This is going to be the first time I’m demonstrating this project in public.

In order to help promoting this event, and also, trying to “lure” you to come to my session if you are based in Melbourne, I’ve recorded a short video demonstrating how I’ve automated the creation of a blank MP and then a Performance Monitor rule (with override) using SharePoint, Orchestrator and SMA. I will also include this same demo in my presentation, and it is probably going to be one of the easier ones Smile.

I’ve uploaded the recording to YouTube, you can watch from https://www.youtube.com/watch?v=aX9oSj_eKeY or from below:

Please watch in Youtube and switch to the full screen mode.

 

If you like what you saw and would like to see more and find out what’s under the hood, please come to this free event next Friday. You can register from here.

image

Creating OpsMgr Instance Group for All Computers Running an Application and Their Health Service Watchers

Written by Tao Yang

OK, the title of this blog is pretty long, but please let me explain what I’m trying to do here. In OpsMgr, it’s quite common to create an instance group which contains some computer objects as well as the Health Service Watchers for these computers. This kind of groups can be used for alert subscriptions, overrides, and also maintenance mode targets.

There are many good posts around this topic, i.e.

From Tim McFadden: Dynamic Computer groups that send heartbeat alerts

From Kevin Holman: Creating Groups of Health Service Watcher Objects based on other Groups

Yesterday, I needed to create several groups that contains computer and health service watcher objects for:

  • All Hyper-V servers
  • All SQL servers
  • All Domain Controllers
  • All ConfigMgr servers

Because all the existing samples I can find on the web are all based on computer names, so I thought I’ll post how I created the groups for above mentioned servers. In this post, I will not go through the step-by-step details of how to create these groups, because depending on the authoring tool that you are using the steps are totally different. But I will go through what the actual XML looks like in the management pack.

Step 1, create the group class

This is straightforward, because this group will not only contain computer objects, but also the health service watcher objects, we must create an instance group.

i.e. Using SQL servers as an example, the group definition looks like this:

  <TypeDefinitions>
    <EntityTypes>
      <ClassTypes>
        <ClassType ID="TYANG.SQL.Server.Computer.And.Health.Service.Watcher.Group" Accessibility="Public" Abstract="false" Base="MSIL!Microsoft.SystemCenter.InstanceGroup" Hosted="false" Singleton="true" />
      </ClassTypes>
    </EntityTypes>
  </TypeDefinitions>

Note: the MP alias “MSIL” is referencing “Microsoft.SystemCenter.InstanceGroup.Library” management pack.

Step 2, Find the Root / Seed Class from the MP for the specific application

Most likely, the application that you are working on (for instance, SQL server) is already defined and monitored by another set of management packs. Therefore, you do not have to define and discover these servers by yourself. The group discovery for the group you’ve just created need to include:

  • All computers running any components of the application (in this instance, SQL Server).
  • And all Health Service Watcher objects for the computers listed above.

In any decent management packs, when multiple application components are defined and discovered, most likely, the management pack author would define a root (seed) class, representing a computer that runs any application components (in this instance, we refer this as the “SQL server”). Once an instance of this seed class is discovered on a computer, there will be subsequent discoveries targeting this seed class that discovers any other application components (using SQL as example again, these components would be DB Engine, SSRS, SSAS, SSIS, etc.).

So in this step, we need to find the root / seed class for this application. Based on what I needed to do, the seed classes for the 4 applications I needed are listed below:

  • SQL Server:
    • Source MP: Microsoft.SQLServer.Library
    • Class Name: Microsoft.SQLServer.ServerRole
    • Alias in my MP: SQL
  • HyperV Server:
    • Source MP: Microsoft.Windows.HyperV.Library
    • Class Name: Microsoft.Windows.HyperV.ServerRole
    • Alias in my MP: HYPERV
  • Domain Controller:
    • Source MP: Microsoft.Windows.Server.AD.Library
    • Class Name: .Windows.Server.AD.DomainControllerRole
    • Alias in my MP: AD
  • ConfigMgr Server
    • Source MP: Microsoft.SystemCenter2012.ConfigurationManager.Library
    • Class Name: Microsoft.SystemCenter2012.ConfigurationManager.Server
    • Alias in my MP: SCCM

Tip: you can use MPViewer to easily check what classes are defined in a sealed MP. Use SQL as example again, in the Microsoft.SQLServer.Library:image

You can easily identify that “SQL Role” is the seed class because it is based on Microsoft.Windows.ComputerRole and other classes use this class as the base class. You can get the actual name (not the display name) from the “Raw XML” tab.

Step 3 Create MP References

Your MP will need to reference the instance group library, as well as the MP of which the application seed class is defined (i.e. SQL library):

image

Step 4 Create the group discovery

The last component we need to create is the group discovery.The Data Source module for the group discovery is Microsoft.SystemCenter.GroupPopulator, and there will be 2 <MembershipRule> sections.i.e. For the SQL group:

 

image

As shown above, I’ve translated each membership rule to plain English. And the XML is listed below. If you want to reuse my code, simply change the line I highlighted in above screenshot to suit your needs.

  <Monitoring>
    <Discoveries>
      <Discovery ID="TYANG.SQL.Server.Computer.And.Health.Service.Watcher.Group.Discovery" Enabled="true" Target="TYANG.SQL.Server.Computer.And.Health.Service.Watcher.Group" ConfirmDelivery="false" Remotable="true" Priority="Normal">
        <Category>Discovery</Category>
        <DiscoveryTypes>
          <DiscoveryRelationship TypeID="MSIL!Microsoft.SystemCenter.InstanceGroupContainsEntities" />
        </DiscoveryTypes>
        <DataSource ID="DS" TypeID="SC!Microsoft.SystemCenter.GroupPopulator">
          <RuleId>$MPElement$</RuleId>
          <GroupInstanceId>$MPElement[Name="TYANG.SQL.Server.Computer.And.Health.Service.Watcher.Group"]$</GroupInstanceId>
          <MembershipRules>
            <MembershipRule>
              <MonitoringClass>$MPElement[Name="Windows!Microsoft.Windows.Computer"]$</MonitoringClass>
              <RelationshipClass>$MPElement[Name="MSIL!Microsoft.SystemCenter.InstanceGroupContainsEntities"]$</RelationshipClass>
              <Expression>
                <Contains>
                  <MonitoringClass>$MPElement[Name="SQL!Microsoft.SQLServer.ServerRole"]$</MonitoringClass>
                </Contains>
              </Expression>
            </MembershipRule>
            <MembershipRule>
              <MonitoringClass>$MPElement[Name="SC!Microsoft.SystemCenter.HealthServiceWatcher"]$</MonitoringClass>
              <RelationshipClass>$MPElement[Name="MSIL!Microsoft.SystemCenter.InstanceGroupContainsEntities"]$</RelationshipClass>
              <Expression>
                <Contains>
                  <MonitoringClass>$MPElement[Name="SC!Microsoft.SystemCenter.HealthService"]$</MonitoringClass>
                  <Expression>
                    <Contained>
                      <MonitoringClass>$MPElement[Name="Windows!Microsoft.Windows.Computer"]$</MonitoringClass>
                      <Expression>
                        <Contained>
                          <MonitoringClass>$Target/Id$</MonitoringClass>
                        </Contained>
                      </Expression>
                    </Contained>
                  </Expression>
                </Contains>
              </Expression>
            </MembershipRule>
          </MembershipRules>
        </DataSource>
      </Discovery>
    </Discoveries>
  </Monitoring>

Result

After I imported the MP into my lab management group, all the SQL computer and Health Service Watcher objects are listed as members of this group:image

Detecting Windows License Activation Status Using ConfigMgr DCM and OpsMgr

Written by Tao Yang

Hello and Happy New year. You are reading my first post in 2015! This is going to a quick post, something I did this week.

Recently, during a ConfigMgr 2012 RAP (Risk and Health Assessment Program) engagement with Microsoft, it has been identified that a small number of ConfigMgr Windows client computers do not have their Windows License activated. The recommendation from the Microsoft ConfigMgr PFE who’s running the RAP was to create a Compliance (DCM) baseline to detect whether the Windows license is activated on client computers.

To respond to the recommendation from Microsoft, I quickly created a DCM baseline with 1 Configuration Item (CI). The CI uses a simple PowerShell script to detect the Windows license status.

image

I configured the CI to only support computers running Windows 7 / Server 2008 R2 and above (as per the minimum supported OS for the SoftwareLicensingProduct WMI class documented on MSDN: http://msdn.microsoft.com/en-us/library/cc534596(v=vs.85).aspx):

image

The CI is configured with 1 compliance rule:

image

Next, I created a Compliance baseline and assigned this CI to it. I then deployed the baseline to an appropriate collection. after few hours, the clients have started receiving the baseline and completed the first evaluation:

SNAGHTMLfeb9db8

Additionally, since I have implemented and configured the latest ConfigMgr 2012 Client MP (Version 1.2.0.0), this DCM baseline assignments on SCOM managed computers are also discovered in SCOM, any non-compliant status would be alerted in SCOM as well.

image

That’s all for today. It is just another example on how to use ConfigMgr DCM, OpsMgr and ConfigMgr 2012 Client MP to quickly implement a monitoring requirement.

This Concludes My Year 2014

Written by Tao Yang

It is one day away from the holiday season of the year. And I have worked HARD over the last few days so I can post my last technical post for the year 2014 before holidays.

First of all, I’d like to wish everyone a Merry Christmas and Happy New Year!

image

2014 has been a fantastic year for me. Here are some of the highlights for me in 2014:

I’ve been awarded as a Microsoft System Center Cloud and Data Center Management MVP for the first time in 1st July 2014.

This is truly my biggest accomplishment of the year. Not to mention being nominated by one of the most well known community leaders in System Center is an accomplishment by itself.

As part of a project team, the project team and I have successfully implemented one of the largest System Center 2012 infrastructures in the country (based on number of seats and number of System Center components implemented).

For those who knows me well, you probably know which one am I talking about Smile.

Had privilege and opportunity to attend Microsoft Global MVP Summit held in Redmond WA in November

I am so glad that I had the opportunity to attend such a wonderful event. Although pretty much everything is under NDA, I can’t really talk about the content of the sessions. I think I can share some pictures here (Some taken from the camera on my phone, some from the SLRs of other SCCDM MVPs).

image

image

image

image

image

image

image

Had opportunities to meet many big names (MVPs and Microsoft employees) in System Center during the MVP summit. Many of those have become good friends too.

I brought a lot of Tim Tam and Kangaroo Jerky to the summit. I didn’t expect Tim Tam to be so popular Smile. If I get awarded again in July 2015, I will make sure I’ll use a bigger suitcase for Tim Tam for the MVP Summit 2015.

image

image

Released a few new and updated Management Packs (ConfigMgr client MP, OpsMgr Self Maintenance MP, SCOM Maintenance Mode Scheduler MP, etc.), OpsMgr dashboards, PowerShell Scripts, SMA Modules etc. to the community.

I’ve lost count, but they should be all on this blog Smile.

Have written 63 blog posts (including this one) in total.

I don’t think the number is very high (only about 5 posts per month), but I’m trying my best Smile. Some of these posts are posting MPs, scripts etc. that I have spent a very long time on. Based on the content, I personally think this is quiet an achievement!

Clocked up over 170,000 hits on this blog in 2014 (to date).

Well, I think I still have a long way to go if comparing with some other popular System Center blogs (Not that I will turn this into a pissing contest). However t is a steady increase from 2013. But I’m sure I’ll do better next year.

What’s Next?

If everything goes as planned, this post will be my final words for 2014. I am taking some time off during the holiday seasons (well, not too long, going back to work on 5th Jan).

During my time off, I will probably spend few days working on an automation solution for OpsMgr – Something I’ve been working on during my spare time since August this year. This leads to the next point.

MVP ComCamp 2015

image

I have been chosen to speak at the MVP Community Camp 2015 in Melbourne on Friday 30th Jan 2015. I will be presenting the topic “Automating SCOM tasks using SMA”. This is something I’ve been working on since August this year. Personally, I think what I have done so far is really cool. This event is going to be held at the Microsoft Melbourne office at Freshwater Place, South Bank, Melbourne. Besides myself, a veteran MVP in System Center ConfigMgr, James Bannan will also deliver a session in Enterprise Mobility Suite in this event.  If you are based in Melbourne, please check out the detail of this event and sessions HERE. I am looking forward to speaking to the Melbourne based System Center folks Smile.

Lastly, I wish everyone have a wonderful time during this holiday season. I will be back in 2015 Smile.

A SMA Integration Module For SharePoint List Operations

Written by Tao Yang

Background

Many Microsoft System Center Orchestrator and Service Management Automation (SMA) users may agree with me, that these two automation platform does not have feature rich end user portals natively. Although System Center Service Manager can be used as a user portal for triggering SCORCH/SMA runbooks, Microsoft SharePoint is also a very good candidate for this purpose.

Integrating SharePoint with Orchestrator and SMA is not something new, many people have done this already. i.e.

System Center Universe America 2014 – Orchestrating Daily Tasks Like a Pro (by Pete Zerger and Anders Bengtsson)

Service Management Automation and SharePoint (by Christian Booth and Ryan Andorfer)

In my opinion, SharePoint (especially SharePoint lists) provides a quick and easy way to setup a web based end user portal for orchestration runbooks. I have also blogged my experiences in the past:

My Experience Manipulating MDT Database Using SMA, SCORCH and SharePoint

SMA Runbook: Update A SharePoint 2013 List Item

To me, not only I am using SharePoint 2013 in my lab; SharePoint Online from my Office 365 subscription, I also have no choice but using SharePoint 2010 in real life.

In my opinion, it is complicated to write SMA runbooks to interact with SharePoint (Using SharePoint web based APIs), not to mention the different versions of SharePoint also dictates how the runbook should be written. It is easier to use Orchestrator as a middle man in between SMA and SharePoint so we can use Orchestrator’s SharePoint integration pack.

Earlier this month, I was developing solutions to use SMA and Azure Automation to create OpsMgr Management Packs catalog on SharePoint 2013 / SharePoint Online sites. I have blogged the 2 solutions here:

On-Premise Solution (SMA + SharePoint 2013)

Cloud Based Solution (Azure Automation + SharePoint Online)

As I mentioned in the previous posts, I had to write a separate SMA module to be used in Azure Automation to interact with SharePoint Online because SharePoint Online sites require a different type of credential (SharePointOnlineCredential) that the PowerShell cmdlet Invoke-RESTMethod does not support. I called that module SharePointOnline back in the previous post and it utilises assemblies from the SharePoint Client Component SDK. I think the SharePoint people also refer to this SDK as Client-Side Object Model (CSOM)

After the MP catalogs posts were published, I have decided to spend a bit more time on the SharePoint Client Component SDK and see if it can help me simplify the activities between SMA and SharePoint. I was really happy to find out, the SharePoint Client Component SDK works for SharePoint 2013, SharePoint Online and SharePoint 2010 (limited). So I have decided to update and extend the original module, making it a generic module for all 3 flavours of SharePoint.

After couple of weeks of coding and testing, I’m pleased to announce the new module is now ready to be released. I have renamed this module to SharePointSDK (Sorry I’m not really creative with names Smile with tongue out).

 

SharePointSDK Module Introduction

The SharePointSDK module contains the following functions:

image

CRUD Operations for SharePoint List items:

Function Description Compatible SharePoint Version
Add-SPListItem Add an item to a SharePoint list 2010, 2013 and SP Online
Get-SPListFields Get all fields of a SharePoint list 2010, 2013 and SP Online
Get-SPListItem Get all list items of a SharePoint list or a specific item by specifying the List Item ID 2010, 2013 and SP Online
Remove-SPListItem Delete an item from a SharePoint list 2010, 2013 and SP Online
Update-SPListItem Update one or more field values of a SharePoint list item 2010, 2013 and SP Online

The functions listed above are the core functionalities this module provides. it provides simplified ways to manipulate SharePoint list items (Create, Read, Update, Delete).

Miscellaneous Functions

Function Description Compatible SharePoint Version
Import-SPClientSDK Load SharePoint Client Component SDK DLLs 2010, 2013 and SP Online
New-SPCredential Based on the type of SharePoint site (On-Prem vs SP Online), create an appropriate credential object to authenticate to the Sharepoint site. 2010, 2013 and SP Online
Get-SPServerVersion Get SharePoint server version 2010, 2013 and SP Online

These functions are called by other functions in the modules. It is unlikely that runbook authors will need to use them directly.

SharePoint List Attachments Operations

Function Description Compatible SharePoint Version
Add-SPListItemAttachment Add an attachment to a SharePoint list item 2013 and SP Online
Get-SPListItemAttachments Download all attached files from a SharePoint list item 2013 and SP Online
Remove-SPListItemAttachment Delete an attached file (based on file name) from a SharePoint list item 2013 and SP Online

As the names suggest, these functions can be used to manage attachments for SharePoint list items.

I’d like to point out  that the Add-SPListItemAttachment function not only support uploading an existing file to the SharePoint list item. it can also be used to create an attachment file directly using a byte array. This function can be used in 3 scenarios:

  • Uploading an existing file from the file system
  • Directly creating a text based file with some contents as a list item attachment.
  • Read the content of an existing binary (or text)  file, save it as a attachment with a different name

 

Configuration Requirements

Download and Prepare the module

The module zip file should consist the following 5 files:

image

  • Microsoft.SharePoint.Client.dll – One of required DLLs from the SDK
  • Microsoft.SharePoint.Client.Runtime.dll – One of required DLLs from the SDK
  • SharePointSDK.psd1 – Module Manifest file
  • SharePointSDK.psm1 – PowerShell module file
  • SharePointSDK-Automation.json – SMA Integration Module Meta File (where the connection asset is defined).

Download SharePointSDK Module

Note:

The zip file you’ve downloaded from the link above DOES NOT contain the 2 DLL files. I am not sure if Microsoft is OK with me distributing their software / intellectual properties. So, just to cover myself, you will need to download the SDK (64-bit version) from Microsoft directly (https://www.microsoft.com/en-us/download/details.aspx?id=35585), install it on a 64-bit computer, and copy above mentioned 2 DLLs into the SharePointOnline module folder.

Once the SDK is installed, you can find these 2 files in “C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\” folder.

Once the DLLs are placed into the folder, zip the SharePointSDK folder to SharePointSDK.zip file again, and the integration module is ready.

image

Import Module

Once the DLLs are zipped into the module zip file, import the module into SMA by using the Import Module button under Assets tab

image

Create a Connection object to the SharePoint site

After the module has been successfully, a connection to SharePoint Site must be created. The Connection type is “SharePointSDK”

image

The following fields must be filled out:

  • Name: Name of the connection.
  • SharePointSiteURL: URL to your sharepoint site
  • UserName : a User who should be part of the site members role (members group have contribute access).
    • If the site is a SharePoint Onine site, this username MUST be in the email address format. (i.e. yourname@yourcompany.com). I believe this account must be an account created in the Office 365 subscription. I have tried using an outlook.com account (added as a SharePoint site member), it didn’t work.
    • When connecting to a On-Prem SharePoint site, you can use the Domain\UserName format (As shown in the screenshot below)
  • Password: Password for the username you’ve specified.
  • IsSharePointOnlineSite: Boolean field (TRUE or FALSE), specify if it is a SharePoint Online site.

i.e. the connection to a SharePoint site in my lab:

image

Sample Runbooks

In order to better demonstrate this module, I have also created 10 sample runbooks:

image

Download Sample runbooks

I’ll now go through each sample runbook.

Runbook: Sample-SPNewUserRequestList

This sample runbook creates a brand new dummy new users requests list on your SharePoint site. The list created by this runbook will then be used by other sample runbooks (for demonstration purposes).

This runbook is expecting 2 input parameters:

  • ListName: The Display Name that you’d like to name the new users requests list (i.e. New Users OnBoarding Requests).
  • SPConnection: The name of the SharePointSDK connection that you’ve created previously (i.e. Based on the connection I’ve created in my lab as shown previously, it is “RequestsSPSite”

image

This runbook creates a list with the following fields:

image

Runbook: Sample-SPGetListFields

This runbook demonstrates how to retrieve all the fields of a particular list.

image

Runbook: Sample-SPAddListItem

This runbook adds an item to the New Users Requests list the previous runbook created. It also demonstrates how to create a text file attachment directly to the list item (without having the need for an existing file on the file system).

It is expecting the following inputs:

  • Title (New users title, i.e. Mr. Dr. Ms, etc)
  • FirstName (New user’s first name)
  • LastName (New user’s last name)
  • Gender (New user’s Gender: Male / Female)
  • UserName (New user’s user vname)
  • AttachmentFileName (file name of the text based attachment)
  • TextAttachmentContent (content of the text file attachment)
  • NewUserListName (display name of the new users requests list. i.e. New Users OnBoarding Requests)
  • SPConnection (The name of the SharePointSDK connection that you’ve created previously (i.e. Based on the connection I’ve created in my lab as shown previously, it is “RequestsSPSite”)

i.e.

image

The list item is created on SharePoint:

SNAGHTML1d5a6df8

Attachment content:

image

Runbook: Sample-SPUpdateListItem

This runbook can be used to update fields of an existing list item on the New Users Requests list.

Runbook: Sample-SPGetAllListItems

This runbook can be used to retrieve ALL items from a list. Each list item are presented as a hash table.

image

Runbook: Sample-SPGetListItem

This runbook can be used to retrieve a single item from a list.

image

Runbook: Sample-SPDeleteListItem

This runbook deletes a single list item by specifying the List Item ID.

Runbook: Sample-SPAddListItemAttachment

This runbook demonstrates 2 scenarios:

  • Directly attaching a file to a list item
  • attach and rename a file to a list item

image

image

Runbook: Sample-SPDeleteListItemAttachments

This runbook demonstrates how to delete an attachment from a list item (by specifying the file name).

Runbook: Sample-SPDownloadListItemAttachments

This runbook demonstrates how to download all files attached to a list item:

image

Files downloaded to the destination folder:

image

Benefit of Using the SharePointSDK Module

Using as a Regular PowerShell Module

As we all know, SMA modules are simply PowerShell modules (sometimes with optional SMA module meta file .json for creating connections). Although this module is primarily written for SMA, it can also be used in other environments such as a regular PowerShell module or in Azure Automation. When using it as a normal PowerShell module, instead of passing the SMA connection name into the functions inside the module, you may provide each individual value separately (Username, password, SharePoint Site URL, IsSharePointOnlineSite).

Simplified scripts to interact with SharePoint

When using this module, most of the operations around the list item only takes very few lines of code.

i.e. Retrieving a list item:

Using PowerShell:

Using PowerShell Workflow (in SMA):

If you use SharePoint 2013’s REST API, the script will be much longer than what I’ve shown above.

Same Code for Different SharePoint Versions

The SharePoint REST API has been updated in SharePoint 2013. Therefore, if we are to use the REST API, the code for Share Point 2013 would look different than SharePoint 2010. Additionally, when throwing SharePoint Online into the mix, as I mentioned previously, it requires different type of credential for authentication, it further complicates the situation if we are to use the REST API. This makes our scripts and runbooks less generic.

By using this SharePointSDK module, I am able to use the same runbooks on SharePoint 2010, 2013 and SharePoint Online sites.

Limitations

During testing, I noticed the 3 attachments related functions in the SharePointSDK module would not work on SharePoint 2010 sites. These functions are:

  • Add-SPListItemAttachment
  • Remove-SPListItemAttachment
  • Get-SPListItemAttachments

After a bit of research, looks like it is a known issue. I didn’t think it too much a big deal because all the core functions (CRUD operations for the list items) work with SharePoint 2010. Therefore, in these 3 functions, I’ve coded a validation step to exit if the SharePoint Server version is below version 15 (SharePoint 2013):

image

Conclusion

If you are using SMA and SharePoint together, I strongly recommend you to download this module and the sample runbooks and give it a try. If you have a look at the sample runbooks, I’m sure you will realise how easy it is to write PowerShell code interacting with SharePoint.

In case you didn’t see the download links, you can download them here:

Download SharePointSDK Module

Download Sample Runbooks

Lastly, I’m not a SharePoint specialist. If you believe I’ve made any mistakes in my code, or there is room for improvement, I’d like to hear from you. Please feel free to drop me an email Smile.

Using Royal TS for PowerShell Remote Sessions

Written by Tao Yang

Background

I have used many Remote Desktop applications in the past. I have to say Royal TS is the one that I like the most! Recently, I showed it to one of my colleagues, after a bit of playing around, he purchased a license for himself too.

Today, my colleague asked me if I knew that Royal TS is also able to run external commands, and he thought it’s pretty cool that he’s able to launch PowerShell in the Royal TS window. Then I thought, if you can run PowerShell in Royal TS, we should be able to establish PS remote sessions in Royal TS too. Within 10 minutes, we managed to create few connections in Royal TS like these:

SNAGHTML1c209a8d

SNAGHTMLa497d178

image

SNAGHTML1c2e5543

In this post, I’ll go through the steps I took to set them up.

Connections to Individual Servers

To create a connection to an individual server,

01. Choose add->External Application:

image

02. Enter the following Details:

Display Name: The name of the server you want to connect to.

Command: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Arguments: -NoExit -Command “Enter-PSSession $CustomField1$”

Working Directory: C:\Windows\System32\WindowsPowerShell\v1.0

On the icon button next to the display name, choose “Use Application Icon” if you want to.

image

image

03. Choose a Credential if you want to connect using an alternative credential

SNAGHTML1c5136c4

If you choose to use an alternative credential,  you must also tick “Use Credentials” box under Advanced tab:

image

04. Enter the remote server name in Custom Field 1:

image

Note: in the arguments field from step 01, I’ve used a Royal TS variable $CustomField1$ as the name of the computer in the Enter-PSSession command. It is more user friendly to use the Custom Field for the computer name, rather than modifying the argument string for each connection that you wish to create.

Create An Ad-Hoc Connection

You can also create a connection in Royal TS for Ad-Hoc connections. In this scenario, you will need to enter the remote computer that you wish to connect to:

image

After the the computer name has been entered, the connection is then established:

image

To create this connection in Royal TS, instead of using the Custom Field 1 for the computer name, I’ve added an additional PowerShell command in the Arguments:

Arguments: -NoExit -Command “$Computer = Read-Host ‘Please enter the Computer Name'; Enter-PSSession $Computer”

image

The Custom Field 1 is no longer required in this scenario. Everything else is the same as the previous sample (for individual computers).

Other Considerations

Maximised PowerShell Window

You may have noticed from the screenshots above, that the PowerShell windows are perfectly fitted in the Royal TS frame. this is because I am also using a customised PS Module that I’ve written in the past to resize the PoewerShell window. Without this module, the PowerShell console would not automatically fit into the Royal TS frame:

image VS image

If you like your console looks like the left one rather than one on the right, please follow the instruction below.

01. Download the PSConsole Module and place it under C:\windows\system32\WindowsPowerShell\v1.0\Modules

image

02. Modify the “All Users Current Host” profile from a normal PowerShell window (NOT within PowerShell ISE). If you are not sure if this profile has been created, run the command below:

image

After the profile is created, open it in notepad (in PowerShell window, type: Notepad $Profile.AllUsersCurrentHost) and add 2 lines of code:

image

After saving the changes, next time when you initiate a connection in Royal TS, the console will automatically maximise to use all the usable space.

Note: Because most likely you will be using an alternative (privileged credential) for these PS remote sessions. therefore the resize console commands cannot be placed into the default profile (current user current host). It must be placed into an All users profile. And also because the resize command only works in a normal PowerShell console (not in PowerShell ISE), therefore the only profile that you can use is the “All Users Current Host” profile from the normal PowerShell console.

Alternatively, if you do not wish to make changes to the All Users Current host profile, you can also add the above mentioned lines into the Royal TS connection arguments field:

i.e.

Arguments: -NoExit -Command “import-module psconsole; resize -max; Enter-PSSession $CustomField1$”

image

Duplicating Royal TS Connections

If you want to create multiple connections, all you need to do is to create the first one manually, and then duplicate it multiple times:

image

When duplicating connections, the only fields you need to change are the Display Name and CustomField1.

WinRM configuration

Needless to say, WinRM must be enabled and properly configured for PS remoting to work. this is a pre-requisite. I won’t go through how to configure WinRM here. Someone actually wrote a whole book on this topic.

Conclusion

I’d like to thank Stefan Koell (blog, twitter), the Royal TS developer (and also my fellow SCCDM MVP) for such an awesome tool. This is now probably THE most used application on all my computers Smile.

If you haven’t tried Royal TS out, please give it a try. Other than the obvious Windows version, there are also a Mac version, an iOS version and an Android version.

A Free Management Pack Catalog for Everyone

Written by Tao Yang

In the 2 most recent post, I have blogged my experience setting up a Microsoft OpsMgr MP catalog using SharePoint and SMA. I managed to produce 2 versions:

  • On-Premise Version: Using SharePoint 2013 and SMA (System Center Service Management Automation)
  • Off-Premise (Cloud) Version: Using Office 365 SharePoint Online and Azure Automation

As I mentioned at the end of the second post, I was working with my SCCDM MVP friend Dan Kregor to create this MP catalog on Sparq Consulting’s public SharePoint site – Free for everyone.

I am pleased to announce, everyone can now access this catalog from this URL:

http://sharepoint.sparqconsulting.com.au/mpcatalog

image

This catalog is publicly available, no login is required. Consider it as a Christmas gift from us Smile.

This catalog is hosted on SharePoint Online, and I have scheduled the Azure Automation runbook to run daily at 9:00pm of my local time to Synchronise with Microsoft’s MP Catalog (It’s Australia Eastern Standard Tiime).

If you like what we’ve done for the System Center community, please help us by spreading the words around, and we will be much appreciated if you want to link to this URL from your websites.

Lastly, other than Dan Kregor, I’d also like to thank all OpsMgr focused MVPs who’s been involved in this discussion since beginning. Thank you for all your input and feedback.

Merry Christmas, everyone.

Using Azure Automation to Build Your Own Microsoft OpsMgr MP Catalog on SharePoint Online

Written by Tao Yang

Background

Previously, I have posted how to build your own Microsoft OpsMgr MP Catalog on SharePoint 2013 using SMA. It is a solution that you can implement on-prem in your own environment if you have existing SharePoint 2013 and SMA infrastructure in place. As I mentioned at the end of the previous post, I have also developed a solution to populate this MP catalog on a Office 365 SharePoint Online site using Azure Automation – a 100% cloud based solution. Because of the differences in APIs between on-prem SharePoint 2013 and SharePoint online, one of the runbooks is completely different than the on-prem version. In this post, I will go through how I’ve setup the MP catalog on SharePoint Online using Azure Automation.

01. Create a List on the SharePoint Online site

The list creation and customization process is exactly the same as the On-Prem SharePoint 2013 version. I won’t go through this again. please refer to Step 1 and the customizing SharePoint List sections in my previous post.

02.  Create a Runbook to Retrieve Microsoft MP info

Again, this runbook is unchanged from the On-Prem version. Simply import it into your Azure Automation account.

Download Get-MSMPCatalog

SNAGHTML803890ed

03. Prepare the SMA Integration Module SharePointOnline

In order to authenticate to SharePoint Online sites, We must use a SharePointOnlineCredentials instance in the script. In my previously post, I wrote a runbook called Populate-OnPremMPCatalog. That runbook utilize Invoke-RestMethod PowerShell cmdlet to interact with SharePoint 2013’s REST API. Unfortunately, we cannot pass a SharePointOnlineCredentials object to this Cmdlet, therefore it cannot be used in this scenario.

Additionally, the SharePointOnlineCredentials class comes from the SharePoint Client Component SDK. In order to create a SharePointOnlineCredentials object in PowerShell scripts, the script need to firstly load the assemblies from 2 DLLs that are part of the SDK. Because I can’t install this SDK in the Azure Automation runbook servers, I needed to figure out a way to be able to load these DLLs in my runbook.

As I have previously written SMA Integration Modules with DLLs embedded in. This time, I figured I can do the same thing – Creating a PowerShell / SMA Integration module that includes the required DLLs. Therefore, I’ve created a customised module in order to load the assemblies. But since the SDK also consists of other goodies, I have written few other functions to perform CRUD (Create, Read, Update, Delete) operations on SharePoint list items. These functions have made the runbook much simpler.

I called this module SharePointOnline, it consists of 5 files:

  • Microsoft.SharePoint.Client.dll – One of required DLLs from the SDK
  • Microsoft.SharePoint.Client.Runtime.dll – One of required DLLs from the SDK
  • SharePointOnline.psd1 – Module Manifest file
  • SharePointOnline.psm1 – PowerShell module file
  • SharePointOnline-Automation.json – SMA Integration Module Meta File (where the connection asset is defined).

image

Download SharePointOnline Module

Note:

The zip file you’ve downloaded from the link above DOES NOT contain the 2 DLL files. I am not sure if Microsoft is OK with 3rd party distributing their software / intellectual properties. So, just to cover myself, you will need to download the SDK (64-bit version) from Microsoft directly (https://www.microsoft.com/en-us/download/details.aspx?id=35585), install it on a 64-bit computer, and copy above mentioned 2 DLLs into the SharePointOnline module folder.

Once the SDK is installed, you can find these 2 files in “C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\” folder.

Once the DLLs are placed into the folder, zip the SharePointOnline folder to SharePointOnline.zip file again, and the integration module is ready.

image

I’d like to also briefly go through this SharePointOnline module. This module contains the following functions:

  • Import-SharePointClientSDK: Load the Assemblies from the 2 DLLs included in the module
  • New-SPOCredential: Create a new SharePointOnlineCredentials object from the username and password provided.
  • Get-SPOListFields: Get all fields from a SharePoint Online list (return an array object)
  • Add-SPOListItem: Add an item to the SharePoint Online list (by passing in a hash table containing the value for each field)
  • Get-SPOListItems: Get all items from a SharePoint Online list (return an array object)
  • Remove-SPOListItem: Remove a list item from a SharePoint Online list (by providing the ID of the item)
  • Update-SPOListItem: Update a list item (by providing the list Item ID and a hash table containing updated values)

This module is made to be re-used for SharePoint Online operations that involves list items. I will write a separate post to go through this in details. But for now, all we need to do is to import it into Azure Automation.

 

04. Import SharePointOnline Module into Azure Automation and Create SharePoint Online Connection

Now that the integration module is ready, it needs to be imported into your Azure Automation account. This is done via the Import Module button under Assets tab.

Once the module is imported, a connection object must also be created.

SNAGHTML806ea00a

image

You must provide the following information when creating the SharePointOnline connection object:

  1. SharePointSiteURL – The URL to your SharePoint Online site (i.e. https//yourcompany.sharepoint.com)
  2. UserName – a User how should be part of the site members role (members group have contribute access). This username MUST be in the email address format. (i.e. yourname@yourcompany.com). I believe this account must be an account created in the Office 365 subscription. I have tried using an outlook.com account (added as a SharePoint site member), it didn’t work.
  3. Password – Password for the username you’ve specified.

i.e.

SNAGHTML80731611

 

05. Create a Runbook to Populate SharePoint List

This is equivalent to the previous runbook Populate-OnPremMPCatalog. I have named it Populate-SPOnlineMPCatalog.

Download Populate-SPOnlineMPCatalog Runbook

SNAGHTML8093a11f

This runbook is expecting 4 parameters:

  • SPOConnection: The name of the SharePointOnline connection that you’ve created earlier.
  • ListName: The list name of your MP catalog list.
  • NotifyByEmail: Specify if you’d like an email notification when new MPs have been added to the catalog.
  • ContactName: If NotifyByEmail is set to “true”, specify the SMAAddressBook connection name for the email notification recipient.

Note: If you’d like to receive email notifications, you also need to import and configure the SendEmail and SendPushNotification modules from my blog. Once the SMTP server connection and the Address book connection are created, please modify line 111 of the runbook with the name of your SMTP server connection:

image

Note: I have previously blogged the issues I have experienced using the SendEmail module in Azure Automation. You may find this post useful: Using the SendEmail SMA Integration Module in Azure Automation.

 

06. Executing Runbook Populate-SPOnlineMPCatalog

When executing the runbook, you need to fill out the parameters listed above:

image

image

Result:

image

Same as the On-Prem version using SMA, you can create a schedule to run this on a regular basis to keep the catalog in sync with Microsoft. I won’t go through the schedule creation again.

Azure Automation Fairshare

Currently, Azure Automation has a “Fairshare” feature, where the maximum allowed execution time for a job is 30 minutes. Fortunately, based on my multiple test runs against multiple Office 365 SharePoint online sites, the first executions of this runbook always complete JUST under 30 minutes. However, if you found your job in Azure Automation is terminated after 30 minutes, you should be able to just run it again to top up the list. But any subsequent runs should only take few minutes.

Conclusion

To me, this post completes the circle. I’m happy that I am able to provide solutions for people who wants to host the catalog On-Premise (by using SharePoint 2013 and SMA), as well as who’d like to hosted in on the cloud (Office 365 and Azure Automation).

The 2 different runbooks (and the additional integration module) are 100% based on what SharePoint are you going to use. There is also a 3rd possible combination: Using SMA to populate SharePoint Online list. In this scenario, the steps are the same as what I described in this post. I have also tested in my lab. it is working as expected.

Additionally, I am also working with my fellow System Center MVP Dan Kregor to make this MP Catalog publicly available for everyone on Sparq Consulting’s public SharePoint Online site. We will make a separate announcement once it is ready. – So even if you can’t setup one up on-prem or on the cloud, we’ve got you covered Smile.

Credit

Thanks for all the System Center MVPs who have provided feedback and input into this solution. Smile

Using SMA to Building Your Own Microsoft OpsMgr Management Pack Catalog On SharePoint 2013

Written by Tao Yang

Background

Over the years that I’ve been working with OpsMgr, for me, the Microsoft Pinpoint MP Catalog was a one stop shop for getting Microsoft management packs. More information about the Pinpoint MP Catalog can be found in Marnix’s post: http://thoughtsonopsmgr.blogspot.com.au/2010/07/pinpoint-management-pack-catalog.html

Based on the information came out in TechEd Europe 2014, looks like there will be changes introduced to the System Center Pinpoint site (http://channel9.msdn.com/Events/TechEd/Europe/2014/Ch9-34, from 09:00 onwards). And it seems the link from Marnix’s post https://pinpoint.microsoft.com/systemcenter is no longer valid.

So I’ve decided to do some experiment, see if I can generate and maintain a Microsoft MP catalog on-premise, for myself – Something I’ve always wanted. It took me couple of days, and I managed to build a MP catalog on a SharePoint 2013 site using PowerShell, SMA, and some existing scripts from the System Center community. The End result looks like this:

SNAGHTML765cc0bc

Currently, this catalog contains 1404 entries, and it is generated by 2 SMA runbooks that I have developed. In this post, I will go through the steps I took to setup this solution.

01. Creating a SharePoint 2013 List

Firstly, I created a list on my SharePoint 2013 site and called it “MP Catalog”:
SNAGHTML7607e74d

image

Once the list is created, go to Site Settings then “Site libraries and lists”

SNAGHTML7609667a

SNAGHTML760b0e10

Choose Customize “MP Catalog” and click on column “Title”

image

Rename this column to “System Name”

image

Add the following additional columns:

  • Categories
  • Catalog Item Id
  • MP Version
  • Public Key
  • Version Independent GUID
  • Download Link
  • Release Date

image

For each of these additional columns, please make sure “Require that this column contains information” is set to “No”.

image

Note: the internal names of these columns would be different than these display names. The script in the SMA runbook will translate these display names to the actual internal names. So it doesn’t matter how you created these columns, as long as the display names are exactly the same as what I listed, it should be OK.

02. Create a Runbook to Retrieve Microsoft OpsMgr Management Packs Info

We have all used the “Download Management Packs” function in the OpsMgr console:

image

Michel Kamp has written a MP that checks updated management packs. This MP utilize the same web service as what “Download Management Packs” wizard uses. I have used some PowerShell code from Michel’s MP in this runbook. –  Thank you, Michel.

I called this runbook Get-MSMPCatalog:

Download Get-MSMPCatalog

SNAGHTML7630c779

This runbook will be called by another runbook, and it returns an arraylist as output.

03. Create a Runbook to Populate the SharePoint List

I created a second runbook to call the first runbook Get-MSMPCatalog, then workout which MPs are not on the SharePoint List, and add the missing ones. I called the second Runbook Populate-OnPremMPCatalog:

Download Populate-OnPremMPCatalog

SNAGHTML763e3f49

As you can see, this runbook is expecting 5 parameters:

  • SharePointSiteURL – URL of the SharePoint site (in my lab, its http://sharepoint01/sites/requests)
  • SavedCredentialName – A credential saved in SMA that has access to the SharePoint site. In my lab, I created an AD service account and assigned it as a member in the SharePoint site.
  • List Name – The list for the MP catalog. In my lab, it’s “MP Catalog”
  • NotifyByEmail – Specify if you’d like an email notification when new MPs have been added to the catalog.
  • ContactName – If NotifyByEmail is set to “true”, specify the SMAAddressBook connection name for the email notification recipient.

Note: If you’d like to receive email notifications, you also need to import and configure the SendEmail and SendPushNotification modules from my blog. Once the SMTP server connection and the Address book connection are created, please modify line 121 of the Populate-OnPremMPCatalog runbook with the name of your SMTP server’s SMA connection:

image

i.e. My SMTP connection and SMAAddress book connection:

SNAGHTML766a1bed

image

04. Execute Runbook Populate-OnPremMPCatalog

When executing this runbook, you will need to fill out some parameters as listed above:

image

image

The first run will take a long time (in my lab, over 1 hour), but any subsequent executions shouldn’t take long at all. i.e. I deleted 204 MPs from the SharePoint list, and execute it again:

image

SNAGHTML76510d6f

I have also created a schedule to execute this runbook daily. This is to make my catalog in sync with Microsoft’s, and notify me when new MPs are released.

image

Customizing the MP Catalog SharePoint List

You may not like the default view of the MP Catalog list. You may want to hide some of the columns. This can be easily done by customising the default view of the list, or creating new views.

image

Conclusion

I’ve always wanted a place where I can simply click on a link to download a particular management pack. I have made this possible by using SMA and SharePoint. The only downside is, only individual management packs are listed. They are not bundled, and no documentations available.

There is also a MP Catalog wiki page on TechNet: https://social.technet.microsoft.com/wiki/contents/articles/16174.microsoft-management-packs.aspx, it is maintained by Microsoft and few System Center MVPs. You should be able to find all the recent MS management packs from there as well.

I have also managed to populate this catalog on a SharePoint Online (Office 365) site using Azure Automation – a 100% cloud based solution. For the cloud based version, one of the runbooks is completely different due to the differences in SharePoint APIs (SharePoint 2013 vs. SharePoint Online). I will post it in few days.

Credit

Thanks for all the System Center MVPs who have provided input and feedback on this topic. You know who you are, much appreciated! Smile

Lastly, please feel free to contact me if you have anything to add on this topic. I’d love to hear from you.