Colourful Norwich skyline illustration

Michael Sage

IT, Digital & Culture

Category: Blog

Posted on

Monitoring ESXi with Nagios

Monitoring ESXi with Nagios

Using the excellent box292_check_vmware plugin and check_esxi_hardware.py script, you can get really great visibility of your ESXi servers.

Notes
For the box293 plugin, you need to have a VMA (details in the manual below).

For the ESXi Hardware you need to install pywbem (apt-get install python-pywbem) and on ESXi 6.5 you need to enable wbem (on the server type: esxcli system wbem set –enable true)

Firewall ports: 443 & 5989 are needed between the monitoring server, VMA and ESXi Hosts

ESXi Checks
Add a server on the VMA – port 443 needs to be open

/usr/lib/vmware-vcli/apps/general/credstore_admin.pl add --server IP.ADDRESS  --username root

Check it works

/usr/lib/nagios/plugins/check_by_ssh -E 1 -l vi-admin -H VMA.IP -C "~/box293_check_vmware.pl --server ESXi.IP --check Host_OS_Name_Version"

Host Datastore Check

/usr/lib/nagios/plugins/check_by_ssh -E 1 -l vi-admin -H VMA.IP -C "~/box293_check_vmware.pl --server ESXi.IP --check Datastore_Usage --name "ESXI Datastore Name" --warning datastore_free:750 --critical datastore_free:700"

$ARG1$ server
$ARG2$ datastore name
$ARG3$ warning Gb
$ARG4$ critical Gb

Host Snapshot(s) Check

/usr/lib/nagios/plugins/check_by_ssh -E 1 -l vi-admin -H VMA.IP -C "~/box293_check_vmware.pl --server ESXi.IP --check Guest_Snapshot --host ESXi.IP --warning snapshot_age:5 --critical snapshot_age:15"

$ARG1$ Server
$ARG2$ Warning
$ARG3$ Critical
Manual
Posted on

Patching a stand alone ESXi Host

Patching a stand alone ESXi Host

Before we start download the latest patch for ESXi from VMware.

The patches are cumulative. I tend to only patch on the major updates.
I am using HP hardware so make sure that you download the custom image if you can. Using the standard image broke my install, thankfully the built in roll back took care of things!
Copy the latest patch to one of your datastores, in the example below I have copied it to a directory called patch on Datastore1.

First check which version of ESXi and patch level you are running: 
~# esxcli system version get
   Product: VMware ESXi
   Version: 6.5.0
   Build: Releasebuild-4564106
   Update: 0
   Patch: 0
Now ensure all the VMs are powered off and the host node is in maintenance mode. Now we query the image profiles that are contained in the patch 
~# esxcli software sources profile list --depot=/vmfs/volumes/datastore1/patch/update-from-esxi6.5.zip

Name                              Vendor        Acceptance Level
--------------------------------  ------------  ----------------
ESXi-6.5.0-20171103001-standard   VMware, Inc.  PartnerSupported
In the HP Custom image there is only one profile, but you may see many options here, unless you know otherwise just use the -standard one. Now we know the name of the profile we can update the host. I recommend doing a dry run first. 
~ #  esxcli software profile update --depot=/vmfs/volumes/datastore1/patch/update-from-esxi6.5.zip --dry-run --profile=ESXi-6.5.0-20171103001-standard

Update Result
   Message: Dryrun only, host not changed. The following installers will be applied: [BootBankInstaller]
   Reboot Required: true
   VIBs Installed:...
   VIBs Removed...
Once you are happy you can run it without the dry run option. 
~ #  esxcli software profile update --depot=/vmfs/volumes/datastore1/patch/update-from-esxi6.5.zip  --profile=ESXi-6.5.0-20171103001-standard

Update Result
   Message: Dryrun only, host not changed. The following installers will be applied: [BootBankInstaller]
   Reboot Required: true
   VIBs Installed:...
   VIBs Removed...
Once is complete you need to reboot the host. 
~ # reboot
When the box is rebooted run the following, to check the new version has applied. 
~ # esxcli system version get
    Product: VMware ESXi
    Version: 6.5.0
    Build: Releasebuild-5310538
    Update: 0
    Patch: 19
If the update fails, simply reboot the server and ESXi will roll back.
Posted on

Hyper-V & OVH – Workgroup

Hyper-V & OVH – Workgroup

On Hyper-V Server

  • Enable Remote Management
  • Enable Remote Desktop
  • Set Computername
  • Enable WSMan and PS-Remoting:
Enable-PSRemoting  
Enable-WSManCredSSP -Role server

On Managing Client

  • Configure hosts or DNS entry for Hyper-V Server
  • Ensure network profile is “Private”:
Set-NetConnectionProfile -InterfaceAlias Ethernet -NetworkCategory Private
  • Temporarily start WinRM and add Hyper-V Server as Trusted Host:
Start-Service -Name winrm  
Set-Item WSMan:\localhost\Client\TrustedHosts -Value "hyperv-server.home.local"  
Stop-Service -Name winrm
  • Enable Hyper-V Management Tools in Windows Features
  • Add Hyper-V Administrator credential with cmdkey:
cmdkey /add:HYPERV-SERVER /user:Administrator /pass:
  • Launch Hyper-V Manager and connect to server (don’t specify user)
    • or, launch Hyper-V Manager with runas:
runas /user:HYPERV-SERVER\Administrator /netonly "mmc virtmgmt.msc"

Summary

After lots of mucking around, this is the bare-minimum working configuration I’ve found to remotely manage a Hyper-V server in a non-domain environment.

I’m still not 100% certain why Hyper-V Manager does not work with supplied credentials and the cmdkey or runas workaround is necessary.

Stolen from: https://blog.ropnop.com/remotely-managing-hyper-v-in-a-workgroup-environment

Posted on

Hyper-V & OVH – Network

Hyper-V & OVH – Network

Network (powershell script)

 

Create an External Switch from the command line

This is the hard part that really had me banging my head of the wall.  If you create an external switch you lose connectivity.

You need to determine the Adapter name to use.  Do to this from a Power Shell prompt run:


Get-NetAdapter

This will list the name of your adapters.  Figure out which one you want to use for your switch.
In my case it was Ethernet 2.

Save the Power Shell Script below to a ps1 file (say c:\MakeSwitch.ps1).  You can run:
Notepad c:\makeswitch.ps1
To do this.  Update it with the correct name determined above.

<#
Make Switch for So You Start Dedicated Server
Usaage:
execute the command Get-NetAdapter
This lists the ethernet adatpers.
Replace “ethernet 2” below with the name of your connected physical adapter
Adapted from:
http://blogs.technet.com/b/heyscriptingguy/archive/2013/10/09/use-powershell-to-create-virtual-switches.aspx
#>

Import-Module Hyper-V
$ethernet = Get-NetAdapter -Name “ethernet 2”
New-VMSwitch -Name externalSwitch -NetAdapterName $ethernet.Name -AllowManagementOS $true -Notes ‘Parent OS, VMs, LAN’
New-VMSwitch -Name privateSwitch -SwitchType Private -Notes ‘Internal VMs only’
New-VMSwitch -Name internalSwitch -SwitchType Internal -Notes ‘Parent OS, and internal VMs’

This actually creates three switches, one of each type.   (external, internal and private).

Posted on

OVH and pfSense

OVH and pfSense

To create a route up to 192.168.23.254 (your main OVH IP), on an interface having no IP in this range, I use the commands: 
route add -net 192.168.23.254/32 -iface em0
route add default 192.168.23.254
The first line tell the firewall that IP address 192.168.23.254 is on the side of the em0 interface (em0 is my WAN interface), the second one use this address as the default gateway. Install shellcmd into pfSense and add the two commands above, this will make it survive a reboot. Adding more IPs Go to the firewall -> virtual IPs Add an IP Alias, add the IPs one at a time with a /32 Your IPs are now ready to use! LAN Internet Firewall -> NAT -> Outbound Manual Outbound NAT rule generation. If it isn’t created automatically add a rule with the Interface of WAN, source of your internal IP (192.168.1.x/24) leave everything else as default and save.
Posted on

Nagios MySQL

Nagios MySQL

Monitoring MySQL servers, we will need to pass through a username and password for MySQL when checking the service availability. Firstly we will create a MySQL user: We start on the MySQL user and we will log in to the database server and create a new user for Nagios to use. I normally use the local check_mysql plugin so there is no need to let the user login from other servers. 
CREATE USER 'nagios'@'localhost' IDENTIFIED BY 'password'
Next check by running the check_mysql plugin 
sudo -s -u nagios /usr/lib/nagios/plugins/check_mysql -H localhost -u nagios -p password
Add this to you nrpe definition file and restart nagios-nrpe-server. Add to your Nagios server and away you go!

Posted on

Auto OpenVPN Script

Auto OpenVPN Script

Build a new VPS and then follow the instructions here! https://github.com/Angristan/OpenVPN-install I had an issue with my install to do with routing, the trouble was in the iptables NAT rule. Check if IPv4 forward is enable. 
# grep net.ipv4.ip_forward /etc/sysctl.conf
net.ipv4.ip_forward=1
# cat /proc/sys/net/ipv4/ip_forward
1
# sysctl -a | grep "net.ipv4.ip_forward "
net.ipv4.ip_forward = 1
If it’s ok, check your iptables rules. 
iptables -t nat -L -vn --line
iptables -L FORWARD -vn --line
I had no nat rule in my nat table, sad times! I added the following and all is well. $IP is your public server IP: 
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -j SNAT --to $IP
Posted on

Cert for ESXi

Cert for ESXi

To generate a certificate request for an ESXi 6.0 host:
 
  1. Open a command prompt and navigate to the OpenSSL directory as previously configured in the Configuring OpenSSL article. By default this is  C:\OpenSSL-Win32\bin.
  2. Run the command:

    openssl req -new -nodes -out rui.csr -keyout rui-orig.key

    This creates the certificate request rui.csr.

  3. Convert the Key to be in RSA format by running these command:

    openssl rsa -in rui-orig.key -out rui.key

Installing and configuring the certificate on the ESXi host 

After the certificate is created, complete the installation and configuration of the certificate on the ESXi 6.0 host:
  1. Navigate to the console of the server to enable SSH on the ESXi 6.0 host.
  2. Log in to the host and then navigate to /etc/vmware/ssl.
  3. Copy the files to a backup location, such as a VMFS volume.
  4. Log in to the host with WinSCP or login locally (my preferred method) and navigate to the /etc/vmware/ssl directory.
  5. Delete the existing  rui.crt and  rui.key from the directory.
  6. Copy the newly created  rui.crt and  rui.key or create them using vi (again my preferred method, I also at the intermediate cert to the .crt file) to the directory using Text Mode or ASCII mode to avoid the issue of special characters (  ^M) appearing in the certificate file.
  7. Type vi rui.crt to validate that there are no extra characters.

    Note: There should not be any erroneous  ^M characters at the end of each line.

  8. Restart the management agents

    /etc/init.d/hostd restart

    /etc/init.d/vpxa restart