OpsMgr 2007 MP Authoring – Proxying Syslog Events

So, I have a windows server receiving ESXi syslog events. However, in my upcoming vCenter Management Pack I wanted events from each host to show up in that hosts’ Event View. Oh no, “it’s impossible!” they said. Poppycock, you just have to use a round-about method (or something simpler that I missed :).

Getting started:

  • I have a server named Win-Syslog, it’s receiving syslog events that show up under it’s Event View per this blog post:  Monitoring ESXi Syslogs with OpsMgr 2007 R2 (Part 1).
  • I have a new host class discovered via SNMP (or my unreleased vCenter MP).
  • Most of the scripts and examples reference my vCenter MP. It’s not done at all, and I can’t promise that it won’t destroy your universe if you import it into OpsMgr. However, so you can poke around to get Syslog Proxying wokring, you can dload the MP here: JPPacks vCenter MP. (Right-click, Save-As).

Custom Data Source

  • Create a custom DataSource with three modules:
    • DataSource: System.ApplicationLog.SysLog.FilteredEventProvider
    • Probe Action: Microsoft.Windows.PowerShellPropertyBagProbe
    • Condition Detection: System.Event.GenericDataMapper
  • The configuration for these should be the same as the post linked above, except that the PowerShell parse script needs to lookup the source ESXi host’s GUID in the OpsMgr database and insert it into the property bag as “CustomMonitoredObjectGUID”. Download and open my vCenter MP for my script. If you want to write this on your own, see my previous post about Looking Up Property GUIDs. Looking up an instance GUID is a similar process.

Custom Write-Action

  • Then, create a custom write-action with the following module:
    • Microsoft.Windows.PowerShellWriteAction
  • Download my vCenter MP for the script.

The Rule

  • Then, create a collection rule that uses these modules:
    • Data Sources: The custom DS made earlier.
    • Actions: The custom WA made earlier.

Once done, all syslog events should now flow from the source GUID of the ESXi host (found by DNS name, or however you configure the ParseScript. Sorry this is choppy and not really a stand-alone post. Developing this vCenter MP is so much fun that it’s giving my personal life quite a beating!

Advertisements

Enable SNMP on ESXi 4.1

Just a quick post. I spent some time trying to figure out how to enable SNMP on ESXi 4.1, and finally found this forum post

Preparation:

  1. Download and Install vSphere CLI. Note, this is different than PowerCLI and the VI Toolkit.

The Process:

  1. Run vSphereCLI and use the following commands:
    1. vicfg-snmp.pl --server <esxiHost> --username root --targets <SNMPReceiverIPAddr>@162/<TrapCommunityString>
    2. vicfg-snmp.pl --server <esxiHost> --username root --communities <ReadCommunityString>
    3. vicfg-snmp.pl --server <esxiHost> --username root --enable
    4. vicfg-snmp.pl --server <esxiHost> --username root --show

References:

Monitoring ESXi Syslogs with OpsMgr 2007 R2 (Part 1)

NOTE: Part 2 – Basic Reports was just published.

NOTE: This is a BETA POST! This post will be updated throughout the next few days to correct spelling mistakes and provide a lot more information about what’s going on in a learn-by-example sort of way. However, I can’t seem to resist the urge of publishing what I already have. Please let me know if you have any problems, and I’ll address them right away! This stuff is _hard_ at first!

There’s a few posts around the net on Syslog monitoring in Operations Manager 2007 ( SCOM ). These posts helped me understand the process (and OpsMgr) a lot better, but didn’t really go as far as I’d like. I needed to capture, process, and export syslog data from my ESXi hosts. Turns out this is extremely difficult to poke around and figure out, but easy to set up if you have a guide.

Some people are okay with the built-in syslog event collectors in OpsMgr, but they give you output like this. Notice how the OpsMgr’s event properties don’t really give you anything useful (Level: Success, Name: Win-Syslog, etc.)

I wanted more, and figured out how to make my events look like this:


Short-cut!

Getting this to work is a long process. If you want results without the learning curve, follow these steps. Otherwise, enter at your own risk!

  1. Complete steps 1-4 under “Preparation” below.
  2. Download and import my custom management pack (right-click, save-as): JPPacks.MonitoringSyslogs 1.0.0.9.
  3. Run these two PowerShell commands on the new VM that will be collecting syslogs:
    New-Item -Path HKLM:\SOFTWARE\JPPacks
    New-ItemProperty -Path HKLM:\SOFTWARE\JPPacks -Name SyslogEnabled -PropertyType DWord -Value 1
  4. “Configure ESXi” according to the screenshots below.
  5. Create an override for the new discovery “JPPacks – Discover Syslog Receivers” changing ‘frequency’ to “20” (seconds).
  6. You should see your VM show up under “Syslog Receiver States” within a minute or so.
  7. Delete that frequency override you made in step 5, or you’ll load down your OpsMgr server.
  8. Send a few test events from Kiwi Syslog Generator. By default only events severity 4 and under (Warning, Error, and Emergency) are logged, this can be changed in the collection rule (see below).

The rest of this post outlines how to create the custom management pack which allows you to parse the raw syslog data into nice, informative events. This includes DNS lookups and mapping <Severity> to <EventLevel> for the pretty Success\Warning\Error icons. Once you have the raw event data in PowerShell and all the modules working together, you can really do anything you want–create new properties, process syslogs differently depending on the source host\IP, etc.

Table of Contents

Preparation

First, prepare your environment:

  1. Deploy a new VM called Win-Syslogs. This system will receive the ESXi syslogs to take the load off of your OpsMgr RMS server.
  2. Install the OpsMgr agent on Win-Syslogs.
  3. Open port 514 in the firewall on Win-Syslogs to your ESXi hosts and your local workstation (for testing).
  4. Download and install Kiwi Syslog Generator on your local workstation (for testing).
  5. Download and install System Center Operations Manager 2007 R2 Authoring Resource Kit on your local workstation.
  6. Download and install XML Notepad 2007 on your local workstation.
  7. Download my Powershell event parsing script from my Chemistry website, and save it as Process-Syslogs-Rev1.ps1.

Configure ESXi

The next step is enabling ESXi Syslogging.

  1. Open your Hosts and Clusters view in vCenter (if you have it).
  2. Select the first host you want to collect from, then click the ‘Configuration’ tab, then ‘Advanced Settings’.
  3. Click ‘Syslog’ on the left pane, then enter the dns name or IP address of Win-Syslogs in the top box. Make sure you’re using Port 514, because OpsMgr will only listen on this port, then click OK.

Creating the Custom Management Pack

Getting Started

  1. First, open System Center Authoring Console and click the ‘New Management Pack’ button.
  2. Then, give your management pack a name.
  3. Next, enter a display name and description, then click Create. The display name will show when looking at the list of management packs in the OpsMgr Console’s Administration view.. The description text is as follows: This management pack contains a custom datasource which processes Syslog events with a PowerShell script, and maps parsed properties to generic OpsMgr events. It also contains a custom rule to collect and publish these events, and a view to look at them.

Creating the Data Source

  1. Next, click the “Type Library” view, then select ‘Module Types’ -> ‘Data Sources’. Right-click and choose “New” -> “Composite Data Source”.
  2. Name your new Data Source, then click “OK”.
  3. Enter a name for your composite data source, then a description. When finished, click the “Member Modules” tab. The name will show up when creating a custom view. The description text here is as follows: This data source recieves a DataItem from the built-in Syslog data source module, processes it with a powershell script using a probe action module, then maps custom parameters from a PowerShell Property Bag to a new Event’s EventData using the Generic Event Mapper condition detection module.
  4. Click “Add.”
  5. Un-Check “Condition Detection” and “Probe Action”, then enter “Syslog” in the “Look for:” text box. Select “System.ApplicationLog.Syslog.FilteredEventProvider”, then enter “SyslogDS” as the Module ID, then click OK.
  6. Click “Configure”.
  7. For Parameter name enter “Severity”, for Operator choose  “Less than or Equal To”, for Value enter “$Config/MinSeverity$”.
  8. The configuration should now say “Port 514”, and have the information filled in that you just entered. If so, click “OK”.
  9. Click ‘Add’ again.
  10. Un-Check “Condition Detection” and “Data Source”, then enter “Powershell” in the “Look for:” text box. Select “Microsoft.Windows.PowerShellPropertyBagProbe”, then enter “ParseScript” as the Module ID, then click OK.
  11. On the ParseScript configuration window, enter “Parse-SyslogEvents.ps1” as the script name, and “30” as the TimeoutSeconds value. Then, click “Edit”.
  12. Familiarize yourself with the default XML structure. You will be adding the script text between <ScriptBody> and </ScriptBody>. You will also be inserting <parameters> tags after </ScriptBody>.
  13. First, insert the Parameters tags as follows:
    <Parameters>
    <Parameter>
    <Name>xmlText</Name>
    <Value>$Data$</Value>
    </Parameter>
    </Parameters>
  14. Next, insert the PowerShell parsing script between the ScriptBody tags as follows:
  15. Save and close the XML file.
  16. Verify that the settings look correct, then click OK.
  17. Click, ‘Add’ again.
  18. Un-Check “Data Source” and “Probe Action”, then enter “Event” in the “Look for:” text box. Select “System.Event.GenericDataMapper”, then enter “Mapper” as the Module ID, then click OK.
  19. On the “Mapper” configuration page, click “Edit”.
  20. Familiarize yourself with the XML structure of the Mapper.
  21. Replace <EventOriginId> through <Params> with the following code:
    <EventOriginId>$Target/Id$</EventOriginId>
    <PublisherId>$MPElement$</PublisherId>
    <PublisherName>$MPElement$</PublisherName>
    <Channel>$Data/Property[@Name="opsmgr_Channel"]$</Channel>
    <LoggingComputer>$Data/Property[@Name="opsmgr_LoggingComputer"]$</LoggingComputer>
    <EventNumber>$Data/Property[@Name="opsmgr_EventNumber"]$</EventNumber>
    <EventCategory>0</EventCategory>
    <EventLevel>$Data/Property[@Name="opsmgr_EventLevel"]$</EventLevel>
    <UserName></UserName>
    <Description>$Data/Property[@Name="Message"]$</Description>
    <Params></Params>

  22. Close the XML file and click “Save”.
  23. Verify that the settings look correct, then click “OK”.
  24. Configure the “NextModule” fields so that SyslogDS leads to ParseScript leads to Mapper leads to Module Output.
  25. Click the “Configuration Schema” tab, then in the bottom frame click “Add”.
  26. Enter “MinSeverity” as the value, then click “OK”.
  27. Change “Type” to “Integer”, then click “Data Types”.
  28. Under the drop-down box, choose “System.Event.Data”, then click “OK”

Creating the Syslog Receiver Class

  1. Click “Service Model” to change views.
  2. Choose “Classes” from the left pane, then Right-Click the right pane and choose “New” -> “Custom Class”.
  3. Enter an ID for your custom class.
    ID: JPPacks.MonitorSyslogs.SyslogReceiver
  4. Enter a name and description for the custom class, then check the “Hosted” box, then Browse for a base class.
    Name: Syslog Receiver
    Description: Computer that receives syslog data.
  5. Click the “List” radio button, then choose “Microsoft.Windows.ComputerRole”, and click “OK”.
  6. Verify that your settings look correct, and click “OK”.

Creating a Discovery for your Class

  1. Click “Health Model” to change views.
  2. Click “Discoveries” in the left pane, then right-click the right pane and choose “New” -> “Registry (Filtered)”.
  3. Enter an ID, Name, and Description for your Discovery. Then, click “…” to browse for a target.
    ID: JPPacks.MonitorSyslogs.DiscoverSyslogReceivers
    Name: JPPacks – Discover Syslog Receivers
    Descriptoion: This discovery looks for the following registry DWORD value: “HKLM\SOFTWARE\JPPacks\SyslogEnabled”. If the value is present and equal to “1”, this discovery rule creates an instance of the ‘Syslog Receiver’ class referencing the discovered computer.
  4. Click the “list” radio button, then in the “Look for” text box enter “windows.o”. Click to choose “Microsoft.Windows.OperatingSystem”, and click “OK”.
  5. Verify that your settings look correct, and click “Next”.
  6. Change the schedule to run once per day. To get things started faster we can create an override inside OpsMgr console once the management pack is imported.
  7. On the “Computer” screen, keep the defaults and click “OK”.
  8. On the registry Probe Configuration page, click “Add”, then “Value”, then enter:
    Name: “SyslogEnabled”
    Path: SOFTWARE\JPPacks\SyslogEnabled
    Attribute Type: Int
  9. Confirm that the settings look correct and click “Next”.
  10. On the “Build Event Expression” page, click “Insert” then choose:
    Parameter Name: “Values/Syslog Enabled”
    Operator: “Equals”
    Value: 1
  11. On the “Discovery Mapper” page, under “Key Properties (Required)”, browse for a property then choose “(Host=Windows Computer)” -> “Principal Name”.
  12. Browse to the same property under “Non Key Properties (Optional)”, then click “Finish”.
  13. Verify that your settings look correct, than click “OK”.
  14. Right-Click your new discovery, and choose “Properties”.
  15. Click the “Configuration” tab, then scroll down to the bottom of the window. Make sure that the last 2 @Type names are set to value “Integer”, than click “OK”.

Reference the Data Warehouse Module

  1. Click “Health Model” to change views.
  2. Click “File” -> “Management Pack Properties”.
  3. Click the “References” tab, then click “Add Reference”.
  4. Browse to the Operations Manager 2007 server directory, and choose “Microsoft.SystemCenter.DataWarehouse.Library.mp”, then click “Open”.
  5. When asked to add the path to your reference paths, click “No”.
  6. Confirm that your settings look OK, then click “Close”.

Creating the Event Collection Rule

  1. First, select “Rules” on the left pane, then right-click the right pane and choose “New” -> “Custom Rule”.
  2. Give your rule an ID, then click OK. I chose “JPPacks.MonitorSyslogs.CollectSyslogs”.
  3. Enter a display name and description for your rule, then change the target to the custom class you created in previous steps. When finished click the “Modules” tab. I chose for following values:
    Name: “JPPacks – Collect Syslogs”
    Description: “This rule collects processed syslog events from the JPPacks custom datasource and publishes them.”
  4. On the “Modules” tab, under “Data Sources”, click “Create”.
  5. Choose the Custom Data Source created in previous steps, then name the Module ID “CustomSyslogDS”, then click “OK”. My Data Source has the ID “JPPacks.MonitorSyslogs.CustomSyslogDS”.
  6. Click “Edit” to configure the CystomSyslogDS module.
  7. Under “MinSeverity”, enter the minimum severity of the Syslog that you want to collect then click, “OK”. A value of “4” will collect all warnings and above (Warnings, Errors, Emergencies, etc.).
  8. On the “Modules” tab, under “Actions”, click “Create”.
  9. In the “Look for:” box, enter “event”, then choose “Microsoft.SystemCenter.CollectEvent”. Then, in the “Module ID” box enter “Collect”, then click “OK”.
  10. On the “Modules” tab, under “Actions”, click “Create” again.
  11. In the “Look for:” box, enter “event”, then choose “Microsoft.SystemCenter.DataWarehouse.PublishEventData”. Then, in the “Module ID” box enter “Publish”, then click “OK”.
  12. Confirm that your settings look correct, then click the “Options” tab.
  13. Uncheck “Confirm Delivery”, and change “Category” to “Event Collection”, then click “OK”.

Creating Views to See Events

  1. Click “Presentation” to switch views.
  2. Click “Views” on the left pane, then right-click the right pane and choose “New” -> “Folder”.
  3. Enter an ID for the view and click “Next”.
    ID: JPPacks.MonitorSyslogs.Views.JPPacksViews
  4. Right-click the right pane again, and choose “New” -> “State View”.
  5. Enter the following information:
    ID: JPPacks.MonitorSyslogs.Views.SyslogReceiversState
    Display Name: JPPacks – Syslog Receivers State
    Description: This view shows the names and states of all discovered Syslog Receivers.
    Target: choose ‘JPPacks.MonitorSyslogs.SyslogReceiver’
    Category: Choose ‘AvailabilityHealth’
  6. Right-click the right pane again, and choose “New” -> “Event View”.
  7. Enter the following information, then click “Finish”.
    ID: JPPacks.MonitorSyslogs.Views.SyslogEvents
    Display Name: JPPacks – Syslog Events
    Description:  This view shows all syslog events received by instances of the ‘Syslog Reveiver’ class.
    Target: choose ‘JPPacks.MonitorSyslogs.SyslogReceiver
  8. Right-click the Event View you created, then choose “Properties”.
  9. Click the “Configuration” tab, then click “Edit”.
  10. Familiarize yourself with the XML layout. We will be replacing the criteria tags.
  11. Replace the <Criteria> tags with the text provided by the file named “JPPacks.MonitorSyslogs.Views.SyslogEvents.txt”. If you followed the Preparation heading,  you already have this. Otherwise, download it here: JPPacks.MonitorSyslogs.Views.SyslogEvents.txt.
  12. Verify that your settings look correct, then click the “Folder” tab.
  13. Ensure that the only box checked is the folder you want this view to reside in. For me, that’s “JPPacks.MonitorySyslogs.Views.JPPacksViews”, then click “OK”.
  14. Right-click the State View created in step 4, and choose “Properties”.
  15. Click the “folder” tab, and ensure that the only box checked is the folder you want this view to reside in. For me, that’s “JPPacks.MonitorySyslogs.Views.JPPacksViews”, then click “OK”.
  16. Right-click the folder you created in step 2 and choose “Properties”.
  17. Enter the following information:
    Name: JPPacks Views
    Description: This folder contains all views related to the JPPacks Management Pack.

Creating a Health Monitor

  1. Click “Monitors” from the left pane, then expand “System.Health.EntityState”. Right-click “System.Health.AvailabilityState” and click “New” -> “Windows Services” -> “Basic Service Monitor”.
  2. Enter the following information then click “Next”.
    Element ID: JPPacks.MonitorSyslogs.Monitors.AvailabilityState.HealthService
    Display Name: JPPacks Monitors – Health Service
    Description: This monitor watches the health service on Syslog Receivers and reports its status.
    Target: choose JPPacks.MonitorSyslogs.SyslogReceiver
    Parent Monitor: choose System.HealthAvailabilityState
    Category: AvailabilityHealth
  3. For “Service Name” enter “HealthService”, then click “Finish”.
  4. Confirm that your settings look correct.

Congrats! Your management pack is complete! Now, follow the steps under “Short-Cut!” to get it working (except step 2 of course).

As always, check my Chemistry Wiki for semi-organized info on Operations Manager as I continue to learn it.

See Part 2 – Basic Reports to get this data exported.

References: