Configuring HP IMC SSH to Switches

I wanted to get our switch configs backed up into HP IMC. Here’s how to enable SSH on the IMC server so that it can contact switches.

By the way, here’s the manual.
HP Intelligent Management Center Base Platform Administrator Guide

  1. Login to IMC and choose System -> Resource Management -> SSH Template List
  2. Click the ‘Modify’ icon in near the right of the row on the ‘default’ template. Alternatively, you can create a new template.
  3. Add your username\password\port info into the template then save.
  4. Navigate to System -> System Configuration -> System Settings.
  5. Scroll down and find ‘External Tool Settings’.
  6. Download plink.exe and psftp.exe from the putty download site, and place them into c:\Program Files\iMC\ on the IMC server itself.
  7. Make sure the following fields are populated with the following info:
    SFTP Client:
    c:\Program Files\iMC\plink.exe
    
    SSH Client
    c:\Program Files\iMC\psftp.exe
  8. Navigate to Resource -> Batch Operation -> SSH Configuration.
  9. Add devices you want to access, then add the SSH information.

That’s it! Everything should work

Advertisements

Installing HP iMC – Intelligent Management Center

I have a single-server HP iMC deployment up and running. Here’s the process:

Overview

  1. Install and Configure the Prereqs (SQL, IIS for redirect, etc).
    1. SQL
    2. AD Bind User
    3. Web Server Certificate
    4. IIS for HTTP Redirection
  2. Install HP iMC.
  3. Configure a custom web server certificate.
  4. Integrate with LDAP and change the initial admin password.
  5. Redirect 80 & 443 to the actual web app’s ports.

Seems simple enough. Let’s get started!

Install and Configure the Prereqs

SQL Config

  • Install SQL and the latest SP’s and CU’s.
  • Enable Mixed-Mode and secure the SA account.
  • Enable TCP/IP.

SA Account

  1. Open SQL Server Management Studio -> right-click your db server -> properties.
  2. Click ‘Security’ then enable ‘SQL Server and Windows Authentication Mode’, then click OK to close the window.
  3. Next, expand the Security folder -> Logins.
  4. Right-click the ‘sa’ account and rename it to ‘sa-imc’ or something similar.
  5. Right-click the ‘sa-imc’ account and choose ‘Properties’.
  6. Change the sa account password.
  7. On the sa account properties page, click ‘status’ -> Enabled.

Enable TCP/IP

  1. Start -> Programs -> SQL Server 2008 -> Configuration Tools -> “SQL Configuration Manager”.
  2. SQL Server Network Configuration -> Protocols -> TCP/IP
  3. Enabled -> Yes
  4. IP Addresses -> 127.0.0.1 -> Enabled: Yes, OK to close the window.
  5. SQL Server Services -> SQL Server -> Right-click and ‘restart’.

AD Bind User

For this, you just need to create an account in AD for iMC to bind with. I recommend using a 64 character random password (at least). Also, I recommend limiting this account such that it cannot interactively login to workstations.

Once created, copy down the distinguishedName attribute. You’ll need this to configure AD integration in iMC.

Web Server Certificate

I wanted a PKI web server certificate for the HP iMC web app. That requires a couple things:

  1. Create a web server certificate template with an exportable key.
  2. Give the iMC server permission to enroll from that template.
  3. Deploy the template to one of your CA’s.
  4. Request the certificate

Creating and Deploying the Template

  1. On a CA, run ‘certtmpl.msc’ to open the Certificate Templates mmc snap-in.
  2. Right-click the ‘Web Server Certificate’ template then choose ‘Duplicate’.
  3. Choose ‘Windows Server 2003’ mode.
  4. Name the new certificate ‘HP iMC Web Server Certificate’.
  5. Under the ‘Request Handling’ tab, choose ‘Allow Private Key to be Exportable’. Also, I chose a key size of 2048.
  6. Under the ‘security’ tab, give the iMC computer account the following permissions: read, enroll.
  7. On your CA, run the ‘Certificate Authority’ snap-in -> Certificate Templates -> Add -> “HP iMC Web Server Certificate’.

Request the Certificate

  1. On your imc server, run ‘mmc’.
  2. Add the certificates snap-in targeted at the computer account.
  3. Navigate to the personal store, right-click -> Request New Certificate.
  4. Click next to select your AD policy, then check the box ‘HP iMC Web Server Certificate’
  5. Under the cert template we’re using, click Details -> Properties.
  6. Leave the ‘subject name’ box completely blank. Under ‘Alternative Subject Name’ add whatever DNS entries you’d like to use for this server.
  7. Under the ‘General’ tab, add a friendly name. It helps organize things quite a bit.
  8. Click OK, then next a couple times to request the cert. Congrats!

Exporting the Cert for use in iMC

HP iMC will not use the certificate by default, so you need to export the certificate with it’s private key. Later on in the process, we’ll convert the PKCS #12 key into a java keystore that HP iMC can accept.

  1. Open MMC -> Add the Certificates Snap-in, targeted at the computer account.
  2. Navigate to the Personal store.
  3. Right click the new Web Server Certificate -> Export.
  4. Export as a PKCS #12, and choose the box to export all certificates in the chain. I recommend using at least a 64 character password. Do NOT choose the box to delete the private key if the export is successful, or certificate will stop working.
  5. Save this file somewhere sensible, like C:\Build on the iMC server.

IIS for Redirection

I wanted to make the system easy for our network guys to access. To me, custom web application ports are difficult to remember — so I installed IIS to redirect all incoming traffic on 80 and 443 to the full https://fqdn:customPort location of iMC.

  1. Install the IIS role with only ‘HTTP Redirection’, logging, and anything else you feel that you need. You don’t need static content, ASP, etc.
  2. Open IIS Manager -> Default Site
  3. Right-click Default Site -> Bindings
  4. Edit the HTTPS binding and select the web server certificate that was requested earlier, then click OK to get back to IIS Manager.
  5. Click Default Site, then on the right-pane double-click “HTTP Redirection”.
  6. Check all three boxes, then for the target location enter:
    https://fqdn.organization.com:8443/imc
  7. Click Save, then exit IIS manager.

Install iMC

  1. Disable UAC and restart. I haven’t been able to get HP iMC working properly with UAC enabled. It’ll install if you run a command prompt as admin, but won’t start properly.
  2. Nagivate to your install media directory and double-click ‘start-install.bat’.
  3. Follow the instructions, using ‘sa-imc’ and the sa password to connect the the SQL server. Everything should just work.
  4. iMC does not automatically start running on boot. To enable auto-load on boot, open the iMC Agent and check the box “Automatically start the services when the OS starts.” You can also start the service immediately if desired.

Convert and Install the Web Server Certificate

The iMC web server is called jserver, which is a java-based web server. It requires a java compatible keystore for certificates. Also, HP iMC can only use a key with the name ‘imc’. To get everything correct we need to convert your PFX file to a java keystore, clone the keychain with a new name, then remove the original keychain with the default name.

  1. Open a command prompt and navigate to C:\Program Files\Java\jre14\bin.
  2. Run the following command to convert the PFX certificate chain exported earlier to a java keystore. It will display an ‘alias’ that looks something like -LongGUID. You’ll need this alias name in the next step.
    keytool.exe -importkeystore -srckeystore C:\build\hp-imc-webservercert-2012.pfx -destkeystore C:\build\keystore -srcstoretype pkcs12 -deststoretype JKS -storepass iMCV300R002 -v
  3. Run the following command to clone the long GUID name into the name ‘imc’ which the web application requires. For <GUID>, use the alias name given in the previous step.
    keytool -keyclone -keystore c:\build\keystore -alias le-hpimcwebserver-03afd524-5d34-4fd0-a2f0-84b1b982afa6 -dest imc
  4. This last command will remove the original alias name from the keystore.
    keytool -delete -keystore c:\build\keystore -alias le-hpimcwebserver-03afd524-5d34-4fd0-a2f0-84b1b982afa6
  5. Now, run the following commands to import the key:
    net stop "HP iMC Server"
    ren "C:\Program Files\iMC\client\security\keystore" "C:\Program Files\iMC\client\security\keystore.orig.bak"
    copy /y "C:\build\keystore" "C:\Program Files\iMC\client\security\"
    net stop "HP iMC Server"
  6. To restart the web server, open the iMC Agent, find the ‘Process’ tab, then restart the ‘jserver.exe’ process.

Change the Admin Password Integrate with LDAP

  1. Navigate to http://localhost. IIS should redirect you to the full https://fqdn/imc and you should _not_ get a certificate warning.
  2. Login as admin\admin.
  3. Click System -> Operator Management -> Operator.
  4. For the account ‘admin’ click ‘Modify’.
  5. Change the password!
  6. Click System -> Operator Management -> Authentication Server.
  7. Fill out the LDAP server information, then click ‘OK’. Make sure to choose ‘Microsoft Active Directory’ for server type. For ‘Admin DN’, enter the distinguishedName of the Bind User created earlier.

Alright! All the basics are now done. Enjoy!

Openfiler Will Not Map iSCSI Luns to iSCSI Targets

NOTE: I found a much better way to do this.

See my new post: Openfiler Will Not Map iSCSI Luns to iSCSI Targets (Part 2). Use the directions below at your own risk! The solution below doesn’t fix the problem for NFS, btrfs, ext2\3\4, or xfs and can possibly lead to data loss.

Last night I was putting the finishing touches on my new Openfiler 2.99 box. Once strange thing, although I could see my volumes in the ‘Volume Management’ page, clicking ‘Map’ in the iSCSI target’s Lun Mapping page would just refresh the page without any changes.

In my case, running lvscan showed that the volumes were ‘inactive’. Changing them to ‘active’ by running ‘lvchange -ay /dev/vol/volname’ allowed me to map again! Sadly, rebooting changed them back to ‘inactive’, probably since they’re on a software raid volume that isn’t automatically assembling on boot.

so, back to editing the rc.local.

Solution

  1. run lvscan to find your inactive volumes.
  2. vi /etc/rc.local and add the following lines (change to your /dev/vg/volume names though):
    service openfiler stop
    #reactive lvm volumes
    lvchange -ay /dev/jetstor/axisdisk
    lvchange -ay /dev/jetstor/dpmbackups
    lvchange -ay /dev/jetstor/desktopimages
    
    #reimport openfiler volumes
    /root/remake_vol_info2
    
    service openfiler start
    

One important note! You might need the remake_vol_info2 script from my previous post here: Openfiler Software Raid Volumes Disappear on Reboot. Because of the software raid problem, my final /etc/rc.local looks like this:

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

#stop openfiler service to make changes to storage subsystem
service openfiler stop

#recreate software raid volumes on JetStor
mdadm -A -s

#recreate volumes in the Openfiler GUI
/root/remake_vol_info2

#reactive lvm volumes
lvchange -ay /dev/jetstor/axisdisk
lvchange -ay /dev/jetstor/dpmbackups
lvchange -ay /dev/jetstor/desktopimages

#restart openfiler service with all volumes
service openfiler start

Using MegaCli to Monitor Openfiler (rev2)

There were a few problems with the last post on monitoring lsi on openfiler. First of all, it wouldn’t send any useful data in the email alerts! For example, it would notify that the online disks were ‘online’, but it doesn’t show any of the offline disks. I was able to find some scripts and modify them to be more useful. Most of this code is from a project called megaraidsas-status, which hasn’t been updated in a while and didn’t work out of the box. I found the project here: HWRaid – LSI MegaRAID SAS.

Configuration Steps

  1. create a file /root/lsi-raidinfo with the following contents:
    #!/usr/bin/python
    
    # megaclisas-status 0.6
    #
    # This program is free software; you can redistribute it and/or modify
    # it under the terms of the GNU General Public License as published by
    # the Free Software Foundation; either version 2 of the License, or
    # (at your option) any later version.
    #
    # This program is distributed in the hope that it will be useful,
    # but WITHOUT ANY WARRANTY; without even the implied warranty of
    # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    # GNU General Public License for more details.
    #
    # You should have received a copy of the GNU General Public License
    # along with Pulse 2; if not, write to the Free Software
    # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
    # MA 02110-1301, USA.
    #
    # Copyright (C) 2007-2009 Adam Cecile (Le_Vert)
    
    ## modified by johnpuskar@gmail.com 08/14/11
    # fixed for LSI 9285-8e on Openfiler
    
    import os
    import re
    import sys
    
    if len(sys.argv) > 2:
        print 'Usage: megaraid-status [-d]'
        sys.exit(1)
    
    # if argument -d, only print disk info
    printarray = True
    printcontroller = True
    if len(sys.argv) > 1:
        if sys.argv[1] == '-d':
            printarray = False
            printcontroller = False
        else:
            print 'Usage: megaraid-status [-d]'
            sys.exit(1)
    
    # Get command output
    def getOutput(cmd):
        output = os.popen(cmd)
        lines = []
        for line in output:
            if not re.match(r'^$',line.strip()):
                lines.append(line.strip())
        return lines
    
    def returnControllerNumber(output):
        for line in output:
            if re.match(r'^Controller Count.*$',line.strip()):
    	    return int(line.split(':')[1].strip().strip('.'))
    
    def returnControllerModel(output):
        for line in output:
            if re.match(r'^Product Name.*$',line.strip()):
    	    return line.split(':')[1].strip()
    
    def returnArrayNumber(output):
        i = 0
        for line in output:
    	if re.match(r'^Virtual Disk.*$',line.strip()):
                i += 1
        return i
    
    def returnArrayInfo(output,controllerid,arrayid):
        id = 'c'+str(controllerid)+'u'+str(arrayid)
        # print 'DEBUG: id = '+str(id)
        operationlinennumber = False
        linenumber = 0
        for line in output:
            if re.match(r'^RAID Level.*$',line.strip()):
                type = 'RAID'+line.strip().split(':')[1].split(',')[0].split('-')[1].strip()
                # print 'debug: type = '+str(type)
            if re.match(r'^Size.*$',line.strip()):
                # Size reported in MB
                if re.match(r'^.*MB$',line.strip().split(':')[1]):
                    size = line.strip().split(':')[1].strip('MB').strip()
                    size = str(int(round((float(size) / 1000))))+'G'
                # Size reported in TB
                elif re.match(r'^.*TB$',line.strip().split(':')[1]):
                    size = line.strip().split(':')[1].strip('TB').strip()
                    size = str(int(round((float(size) * 1000))))+'G'
                # Size reported in GB (default)
                else:
                    size = line.strip().split(':')[1].strip('GB').strip()
                    size = str(int(round((float(size)))))+'G'
            if re.match(r'^State.*$',line.strip()):
                state = line.strip().split(':')[1].strip()
            if re.match(r'^Ongoing Progresses.*$',line.strip()):
    	    operationlinennumber = linenumber
            linenumber += 1
            if operationlinennumber:
                inprogress = output[operationlinennumber+1]
            else:
                inprogress = 'None'
        return [id,type,size,state,inprogress]
    
    def returnDiskInfo(output,controllerid,currentarrayid):
        arrayid = False
        oldarrayid = False
        olddiskid = False
        table = []
        state = 'Offline'
        model = 'Unknown'
        enclnum = 'Unknown'
        slotnum = 'Unknown'
        enclsl = 'Unknown'
    
        firstDisk = True
        for line in output:
            if re.match(r'Firmware state: .*$',line.strip()):
                state = line.split(':')[1].strip()
            if re.match(r'Slot Number: .*$',line.strip()):
                slotnum = line.split(':')[1].strip()
            if re.match(r'Inquiry Data: .*$',line.strip()):
                model = line.split(':')[1].strip()
                model = re.sub(' +', ' ', model)
            if re.match(r'Enclosure Device ID: [0-9]+$',line.strip()):
                enclnum = line.split(':')[1].strip()
                if firstDisk == True:
                    firstDisk = False
                else:
                    enclsl = str(enclnum)+':'+str(slotnum)
                    table.append([str(enclsl), model, state])
        # Last disk of last array
        enclsl = str(enclnum)+':'+str(slotnum)
        table.append([str(enclsl), model, state])
        arraytable = []
        for disk in table:
            arraytable.append(disk)
        return arraytable
    
    cmd = '/opt/MegaRAID/MegaCli/MegaCli64 -adpCount -NoLog'
    output = getOutput(cmd)
    controllernumber = returnControllerNumber(output)
    
    bad = False
    
    # List available controller
    if printcontroller:
        print '-- Controllers --'
        print '-- ID | Model'
        controllerid = 0
        while controllerid < controllernumber:
            cmd = '/opt/MegaRAID/MegaCli/MegaCli64 -AdpAllInfo -a'+str(controllerid)+' -NoLog'
            output = getOutput(cmd)
            controllermodel = returnControllerModel(output)
            print 'c'+str(controllerid)+' | '+controllermodel
            controllerid += 1
        print ''
    
    if printarray:
        controllerid = 0
        print '-- Volumes --'
        print '-- ID | Type | Size | Status | InProgress'
        # print 'controller number'+str(controllernumber)
        while controllerid < controllernumber:
            arrayid = 0
    	cmd = '/opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -lall -a'+str(controllerid)+' -NoLog'
    	output = getOutput(cmd)
    	arraynumber = returnArrayNumber(output)
    	# print 'array number'+str(arraynumber)
            while arrayid             cmd = '/opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -l'+str(arrayid)+' -a'+str(controllerid)+' -NoLog'
    #            print 'DEBUG: running '+str(cmd)
    	    output = getOutput(cmd)
    	    # print 'DEBUG: output '+str(output)
                arrayinfo = returnArrayInfo(output,controllerid,arrayid)
    	    print 'volinfo: '+arrayinfo[0]+' | '+arrayinfo[1]+' | '+arrayinfo[2]+' | '+arrayinfo[3]+' | '+arrayinfo[4]
                if not arrayinfo[3] == 'Optimal':
                    bad = True
    	    arrayid += 1
            controllerid += 1
        print ''
    
    print '-- Disks --'
    print '-- Encl:Slot | Model | Status'
    
    controllerid = 0
    while controllerid < controllernumber:
        arrayid = 0
        cmd = '/opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -lall -a'+str(controllerid)+' -NoLog'
        output = getOutput(cmd)
        arraynumber = returnArrayNumber(output)
        while arrayid         #grab disk arrayId info
            #cmd = '/opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -a'+str(controllerid)+' -NoLog'
            cmd = '/opt/MegaRAID/MegaCli/MegaCli64 -PDList -a'+str(controllerid)+' -NoLog'
            #print 'debug: running '+str(cmd)
            output = getOutput(cmd)
            arraydisk = returnDiskInfo(output,controllerid,arrayid)
    
            for array in arraydisk:
                print 'diskinfo: '+array[0]+' | '+array[1]+' | '+array[2]
    	arrayid += 1
        controllerid += 1
    
    if bad:
        print '\nThere is at least one disk/array in a NOT OPTIMAL state.'
        sys.exit(1)
    
  2. chown 700 /root/lsi-raidinfo
  3. create a file /root/lsi-checkraid with the following contents:
    #!/usr/bin/python
    
    # created by johnpuskar@gmail.com on 08/14/11
    # rev 01
    
    import os
    import re
    import sys
    
    if len(sys.argv) > 1:
      print 'Usage: accepts stdin from lsi-raidinfo'
      sys.exit(1)
    
    blnBadDisk = False
    infile = sys.stdin
    for line in infile:
      #print 'DEBUG!! checking line:'+str(line)
      if re.match(r'diskinfo: .*$',line.strip()):
        if re.match(r'^((?!Online, Spun Up|Online, Spun down|Hotspare, Spun Up|Hotspare, Spun down).)*$',line.strip()):
          blnBadDisk = True
          #print 'DEBUG!! bad disk found!'
      if re.match(r'volinfo: ',line.strip()):
        if re.match(r'^((?!Optimal).)*$',line.strip()):
          #print 'DEBUG!! bad vol found!'
          blnBadDisk = True
    
    if blnBadDisk == True:
      print 'RAID ERROR'
    else:
      print 'RAID CLEAN'
    
  4. chown 700 /root/lsi-checkraid
  5. create a file /root/lsi-emailalerts with the following contents
    #!/bin/sh
    
    #get raid status info
    /root/lsi-raidinfo > /tmp/lsi-raidinfo.txt
    cat /tmp/lsi-raidinfo.txt | /root/lsi-checkraid > /tmp/lsi-checkraid.txt
    
    #check raid status info
    if grep -qE "RAID ERROR" /tmp/lsi-checkraid.txt
    then
    /opt/MegaRAID/MegaCli/MegaCli64 -PdList -aALL > /tmp/lsi-megaCLIdump.txt
    cat /tmp/lsi-raidinfo.txt | mailx -s "Warning: HOSTNAME failed disk or degraded array" MAILTOADDR -r MAILFROMADDR -a /tmp/lsi-megaCLIdump.txt
    fi
    
    rm -f /tmp/lsi-megaCLIdump.txt
    rm -f /tmp/lsi-raidinfo.txt
    rm -f /tmp/lsi-checkraid.txt
    exit 0
    
  6. modify the mailx line in /root/emailalerts with the correct from/to/subject
  7. chown 700 /root/lsi-emailalerts
  8. run crontab -e and add the following line:
    */5 * * * * /root/lsi-emailalerts
Now, when there’s a volume not ‘optimal’ or a disk not not ‘hotspare’ or ‘online, spun up’. You’ll get an email with useful info!

Example Email

subject: “Warning: HOSTNAME failed disk or degraded array”

attachment: lsi-megaCLIdump.txt

message:

— Controllers —
— ID | Model
c0 | LSI MegaRAID SAS 9285-8e

— Volumes —
— ID | Type | Size | Status | InProgress
volinfo: c0u0 | RAID6 | 57298G | Optimal | None

— Disks —
— Encl:Slot | Model | Status
diskinfo: 32:1 | SEAGATE ST33000650SS 0002Z2902LAT | Online, Spun down
diskinfo: 32:2 | SEAGATE ST33000650SS 0002Z2905B0E | Online, Spun down
diskinfo: 32:3 | SEAGATE ST33000650SS 0002Z2905ANS | Online, Spun down
diskinfo: 32:4 | SEAGATE ST33000650SS 0002Z2903DM0 | Online, Spun down
diskinfo: 32:5 | SEAGATE ST33000650SS 0002Z2905HE4 | Online, Spun down
diskinfo: 32:6 | SEAGATE ST33000650SS 0002Z2905B3F | Online, Spun down
diskinfo: 32:7 | SEAGATE ST33000650SS 0002Z2903DVH | Online, Spun down
diskinfo: 32:8 | SEAGATE ST33000650SS 0002Z2905B1W | Online, Spun down
diskinfo: 32:9 | SEAGATE ST33000650SS 0002Z29040GF | Online, Spun down
diskinfo: 32:10 | SEAGATE ST33000650SS 0002Z29032B5 | Online, Spun down
diskinfo: 32:11 | SEAGATE ST33000650SS 0002Z2905C23 | Online, Spun down
diskinfo: 32:12 | SEAGATE ST33000650SS 0002Z2904RMH | Online, Spun down
diskinfo: 32:13 | SEAGATE ST33000650SS 0002Z29035VM | Online, Spun down
diskinfo: 32:14 | SEAGATE ST33000650SS 0002Z2905H0C | Online, Spun down
diskinfo: 32:15 | SEAGATE ST33000650SS 0002Z29031SY | Online, Spun down
diskinfo: 32:16 | SEAGATE ST33000650SS 0002Z29031ZZ | Online, Spun down
diskinfo: 32:17 | SEAGATE ST33000650SS 0002Z2905AVN | Online, Spun down
diskinfo: 32:18 | SEAGATE ST33000650SS 0002Z2905DW9 | Online, Spun down
diskinfo: 32:19 | SEAGATE ST33000650SS 0002Z2905B2E | Online, Spun down
diskinfo: 32:20 | SEAGATE ST33000650SS 0002Z2903DP9 | Online, Spun down
diskinfo: 32:21 | SEAGATE ST33000650SS 0002Z2903YTQ | Online, Spun down
diskinfo: 32:22 | SEAGATE ST33000650SS 0002Z2906NEL | Online, Spun down
diskinfo: 32:23 | SEAGATE ST33000650SS 0002Z2906NMY | Online, Spun down
diskinfo: 32:24 | SEAGATE ST33000650SS 0002Z29035RL | Hotspare, Spun Up

Using MegaCLI to Monitor Openfiler

UPDATE — There’s a better way to do this. See my latest post here: Using MegaCLI to Monitor Openfiler (rev2).

There’s quite a few posts around the ‘net on Openfiler and MegaCLI. Why another one? I wanted to customize my scripts a bit differently, and none of them worked 100% for me. Most of the code is scavenged but attributed where possible.

Installing MegaCLI

  1. First, find and download the latest MegaCLI supported by your controller on the LSI website.
  2. Extract the RPM’s on your workstation and scp them to your openfiler box to /root (on windows, WinSCP works).
  3. SSH to the openfiler box
  4. /root/rpm2cpio MegaCli-8.00.11-1.i386.rpm  | cpio -idmv
  5. /root/rpm2cpio Lib_Utils-1.00-05.noarch.rpm   | cpio -idmv
  6. Test with /opt/MegaRAID/MegaCLI/MegaCli64 -PDList -aAll

Configuring MegaCLI and Cron

  1. vi /opt/MegaRAID/MegaCLI/analysis.awk
  2. Add the following to this new file:
    # This is a little AWK program that interprets MegaCLI output
    #using two spaces at the beginning of each line for outlook -- see http://stackoverflow.com/questions/247546/outlook-autocleaning-my-line-breaks-and-screwing-up-my-email-format
    #ref: http://timjacobs.blogspot.com/2008/05/installing-lsi-logic-raid-monitoring.html
    /Device Id/ { counter += 1; device[counter] = $3 }
    /Firmware state/ { state_drive[counter] = $3 }
    /Inquiry/ { name_drive[counter] = $3 " " $4 " " $5 " " $6 }
    END {
    for (i=1; i
  3. vi /root/raidstatus
  4. Add the following to this new file. NOTE: make sure to change SYSTEMNAME, and the target\sender addresses.
    #!/bin/sh
    #ref: http://timjacobs.blogspot.com/2008/05/installing-lsi-logic-raid-monitoring.html
    /opt/MegaRAID/MegaCli/MegaCli64 -PdList -aALL > /tmp/SYSTEMNAME-megaCLIdump.txt
    cat /tmp/SYSTEMNAME-megaCLIdump.txt | awk -f /opt/MegaRAID/MegaCli/analysis.awk > /tmp/megarc.raidstatus
    
    #ref: https://forums.openfiler.com/viewtopic.php?id=4711 (fishguy; post 6)
    cd /opt/MegaRAID/MegaCli
    ./MegaCli64 -AdpAllInfo -aALL | grep "Degraded" > degraded.txt
    ./MegaCli64 -AdpAllInfo -aALL | grep "Failed Disks" > failed.txt
    a='cat degraded.txt'
    b='cat failed.txt'
    echo $a $b | grep "1" > dead.txt
    if [[ $? -eq 0 ]];
    then
    cat /tmp/megarc.raidstatus | mailx -s "Warning: SYSTEMNAME.chem failed disk or degraded array" alerts@email.com -r sender@email.com -a /tmp/SYSTEMNAME-megaCLIdump.txt
    fi
    
    rm -f /opt/MegaRAID/MegaCli/degraded.txt
    rm -f /opt/MegaRAID/MegaCli/failed.txt
    rm -f /opt/MegaRAID/MegaCli/dead.txt
    rm -f /tmp/SYSTEMNAME-megaCLIdump.txt
    rm -f /tmp/megarc.raidstatus
    exit 0
    
    
  5. chmod 700 /root/raidstatus
  6. Next, open the Openfiler GUI, click the “System” tab, then click “Notifications” in the right navigation panel.
  7. Enter your notifications (email) settings into this configuration page and click “save”. My emails from the command line wouldn’t send until I completed this step.
  8. Next run “chrontab -e” and add the following line:
  9. */5 * * * * /root/raidstatus

Once complete, cron should run this script every 5mins and email if any raid members are offline.

Reference:

OpsMgr Errors – AD_Client_Update_DCs.vbs

Got this alert on one of the machines I have configured for AD client monitoring.

Error Text:
Command executed: “C:\Windows\system32\cscript.exe” /nologo “AD_Client_Update_DCs.vbs” winfs.chemistry.ohio-state.edu CHEMISTRY false 3 {ABFB3D66-E484-7150-CAB9-0901473EEC93}
Working Directory: C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 72\541\
One or more workflows were affected by this.
Workflow name: AD_Client_Update_DCs
Instance name: AD Client Monitoring

Resolution:

  1. Open Authoring -> Rules
  2. Override the rule, “AD Client Updates DCs”. Add any of your domain controllers to the ‘domain controller’ field, and set ‘Site Discovery Mode’ to 3.

References: