Colourful Norwich skyline illustration

Michael Sage

IT, Digital & Culture

Extend a partition and LVM with Ubuntu 16.04

There is a risk of data loss doing this!!!

First add the additional disk space using your virtualisation admin software (i.e. ESXi / Proxmox / etc)

Rescan the bus

echo 1 > /sys/class/block/sda/device/rescan

Next start fdisk

sudo fdisk /dev/sda

Press p to print the current partition list. Copy the start block for both sda2 and sda5. Now we need to delete the partitions.

Press d to delete the partion accept the default of ‘2’
Press d again and accept the default of ‘5’

Now press n for a new partition

Create an extended partition, make sure the start block is that of your “old” sda2 partition.

Accept the other defaults to use all available space.

Now press n again

This time accept the defaults. The start block will be wrong, but this is ok and a slight anomaly with this method.

This bit is super, world ending, important.

Once you are back to the fdisk prompt press x (to enter expert mode)

Press b and make sure sda5 is selected.

Enter the start value that you copied earlier for sda5.

Once you are back at the expert command prompt, press r (to return to the main menu) and then w (to write the changes and exit.

We’ve now finished with fdisk.

Now sync the changes with the running OS.

partprobe

Extending LVM

Run

pvresize /dev/sda5
lvextend -l +100%FREE /dev/VGNAME/LVNAME
resize2fs /dev/VGNAME/LVNAME

If you don’t know your VGNAME or LVNAME run

lvdisplay

That’s all there is to it!

Expand a hard disk with LVM

The “hardware” part, “physically” adding diskspace to your VM

Increasing the disk size can be done via the vSphere Client, by editing the settings of the VM (right click > Settings).

If the “Provisioned Size” area (top right corner) is greyed out, consider turning off the VM first (if it does not allow “hot adding” of disks/sizes), and check if you have any snapshots made of that VM. You can not increase the disk size, as long as there are available snapshots.

Partitioning the unallocated space: if you’ve increased the disk size

Once you’ve changed the disk’s size in VMware, boot up your VM again if you had to shut it down to increase the disk size in vSphere. If you’ve rebooted the server, you won’t have to rescan your SCSI devices as that happens on boot. If you did not reboot your server, rescan your SCSI devices as such.

Then rescan the bus.

~$ 'echo 1 > /sys/class/block/sda/device/rescan'

Create the new partition

Once the rescan is done (should only take a few seconds), you can check if the extra space can be seen on the disk.

~$  fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         391     3036285   8e  Linux LVM

So the server can now see the 10GB hard disk. Let’s create a partition, by start fdisk for the /dev/sda device.

~$  fdisk /dev/sda

The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): n

Now enter ‘n’, to create a new partition.

Command action
e   extended
p   primary partition (1-4)
 p

Now choose “p” to create a new primary partition. Please note, your system can only have 4 primary partitions on this disk! If you’ve already reached this limit, create an extended partition.

Partition number (1-4): 3

Choose your partition number. Since I already had /dev/sda1 and /dev/sda2, the logical number would be 3.

First cylinder (392-1305, default 392): 
Using default value 392
Last cylinder or +size or +sizeM or +sizeK (392-1305, default 1305): 
Using default value 1305

Note; the cylinder values will vary on your system. It should be safe to just hint enter, as fdisk will give you a default value for the first and last cylinder (and for this, it will use the newly added diskspace).

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 8e
Changed system type of partition 3 to 8e (Linux LVM)

Now type t to change the partition type. When prompted, enter the number of the partition you’ve just created in the previous steps. When you’re asked to enter the “Hex code”, enter 8e, and confirm by hitting enter.

Command (m for help): w

Once you get back to the main command within fdisk, type w to write your partitions to the disk. You’ll get a message about the kernel still using the old partition table, and to reboot to use the new table. The reboot is not needed as you can also rescan for those partitions using partprobe. Run the following to scan for the newly created partition.

~$ partprobe -s

If that does not work for you, you can try to use “partx” to rescan the device and add the new partitions. In the command below, change /dev/sda to the disk on which you’ve just added a new partition.

~$ partx -v -a /dev/sda

If that still does not show you the newly created partition for you to use, you have to reboot the server. Afterwards, you can see the newly created partition with fdisk.

~$  fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         391     3036285   8e  Linux LVM
/dev/sda3             392        1305     7341705   8e  Linux LVM

Extend your Logical Volume with the new partition

Now, create the physical volume as a basis for your LVM. Please replace /dev/sda3 with the newly created partition.

~$  pvcreate /dev/sda3
Physical volume "/dev/sda3" successfully created

Now find out how your Volume Group is called.

~$  vgdisplay
--- Volume group ---
VG Name               VolGroup00
...

Let’s extend that Volume Group by adding the newly created physical volume to it.

~$  vgextend VolGroup00 /dev/sda3
Volume group "VolGroup00" successfully extended

With pvscan, we can see our newly added physical volume, and the usable space (7GB in this case).

~$  pvscan
PV /dev/sda2   VG VolGroup00   lvm2 [2.88 GB / 0    free]
PV /dev/sda3   VG VolGroup00   lvm2 [7.00 GB / 7.00 GB free]
Total: 2 [9.88 GB] / in use: 2 [9.88 GB] / in no VG: 0 [0   ]

Now we can extend Logical Volume (as opposed to the Physical Volume we added to the group earlier). The command is “lvextend /dev/VolGroupxx /dev/sdXX“.

~$  lvextend /dev/VolGroup00/LogVol00 /dev/sda3
Extending logical volume LogVol00 to 9.38 GB
Logical volume LogVol00 successfully resized

If you’re running this on Ubuntu, use the following.

~$  lvextend /dev/mapper/vg-name /dev/sda3

All that remains now, it to resize the file system to the volume group, so we can use the space. Replace the path to the correct /dev device if you’re on ubuntu/debian like systems.

~$  resize2fs /dev/VolGroup00/LogVol00
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/VolGroup00/LogVol00 is mounted on /; on-line resizing required
Performing an on-line resize of /dev/VolGroup00/LogVol00 to 2457600 (4k) blocks.
The filesystem on /dev/VolGroup00/LogVol00 is now 2457600 blocks long.

And we’re good to go!

~$  df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 9.1G 1.8G  6.9G  21% /
/dev/sda1              99M   18M   77M  19% /boot
tmpfs                 125M     0  125M   0% /dev/shm

Unifi Video Mongodb remove

Occasionally you hit an issue with a unifi camera and it won’t connect or remove from the controller. The easiest / only way to fix this is to manually remove it from the database.

SSH into the NVR and find all the cameras:

mongo localhost:7441/av
db.camera.find()
db.camera.find({}, {_id: 0, name: 1, uuid: 2})

Look for the camera in question, we will want the UUID

"Driveway", "uuid" : "1fbfb420-a091-3c8f-b184-e43ec862b14a"

Then remove it, exit the tool, and restart UFV

db.camera.remove({uuid:"1fbfb420-a091-3c8f-b184-e43ec862b14a"})
exit
service unifi-video stop; service unifi-video start

OPNSense & Duo

OPNSense & Duo

Set the OPNSense LDAP connection to use the DN rather than the domain\user format or DUO fails to recognise it as the same user. By default the Authentication Proxy doesn’t require 2FA for the first bind in a connection. This is to support systems that bind as a service account, search for the user account, and then bind as the user. It looks like your system may connect and bind as the service account, then disconnects, then connects again to bind as the end user. Look at the exempt_primary_bind and exempt_ou_1 options  and try settingexempt_primary_bind=false and exempt_ou_1=the DN of the service account.

OPNSense & OVH/SyS

This is much simpler in OPNSense

Add gateway

System -> Gateways -> All

Add Gateway

Change Interface to WAN.

Add your primary OVH IP gateway in the “Gateway” field (this will almost certainly not be the gateway for your IP address block) and tick the “far gateway” box.

Hit save and then reload

Adding more IPs

Go to the firewall -> virtual IPs

Add an IP Alias, add the IPs one at a time with a /32

Your IPs are now ready to use!

LAN Internet

Firewall -> NAT -> Outbound

Manual Outbound NAT rule generation. If it isn’t created automatically add a rule with the Interface of WAN, source of your internal IP (192.168.1.x/24) & any VPN IP leave everything else as default and save. Check that your WAN address is chosen rather than interface address (or your outgoing traffic will come from any of your external IPs)

Monitoring ESXi with Nagios

Monitoring ESXi with Nagios

Using the excellent box292_check_vmware plugin and check_esxi_hardware.py script, you can get really great visibility of your ESXi servers.

Notes
For the box293 plugin, you need to have a VMA (details in the manual below).

For the ESXi Hardware you need to install pywbem (apt-get install python-pywbem) and on ESXi 6.5 you need to enable wbem (on the server type: esxcli system wbem set –enable true)

Firewall ports: 443 & 5989 are needed between the monitoring server, VMA and ESXi Hosts

ESXi Checks
Add a server on the VMA – port 443 needs to be open

/usr/lib/vmware-vcli/apps/general/credstore_admin.pl add --server IP.ADDRESS  --username root 

Check it works

/usr/lib/nagios/plugins/check_by_ssh -E 1 -l vi-admin -H VMA.IP -C "~/box293_check_vmware.pl --server ESXi.IP --check Host_OS_Name_Version"

Host Datastore Check

/usr/lib/nagios/plugins/check_by_ssh -E 1 -l vi-admin -H VMA.IP -C "~/box293_check_vmware.pl --server ESXi.IP --check Datastore_Usage --name "ESXI Datastore Name" --warning datastore_free:750 --critical datastore_free:700"

$ARG1$ server
$ARG2$ datastore name
$ARG3$ warning Gb
$ARG4$ critical Gb

Host Snapshot(s) Check

/usr/lib/nagios/plugins/check_by_ssh -E 1 -l vi-admin -H VMA.IP -C "~/box293_check_vmware.pl --server ESXi.IP --check Guest_Snapshot --host ESXi.IP --warning snapshot_age:5 --critical snapshot_age:15"

$ARG1$ Server
$ARG2$ Warning
$ARG3$ Critical

Patching a stand alone ESXi Host

Patching a stand alone ESXi Host

Before we start download the latest patch for ESXi from VMware.

The patches are cumulative. I tend to only patch on the major updates.
I am using HP hardware so make sure that you download the custom image if you can. Using the standard image broke my install, thankfully the built in roll back took care of things!
Copy the latest patch to one of your datastores, in the example below I have copied it to a directory called patch on Datastore1.

First check which version of ESXi and patch level you are running:
~# esxcli system version get
   Product: VMware ESXi
   Version: 6.5.0
   Build: Releasebuild-4564106
   Update: 0
   Patch: 0
Now ensure all the VMs are powered off and the host node is in maintenance mode. Now we query the image profiles that are contained in the patch
~# esxcli software sources profile list --depot=/vmfs/volumes/datastore1/patch/update-from-esxi6.5.zip

Name                              Vendor        Acceptance Level
--------------------------------  ------------  ----------------
ESXi-6.5.0-20171103001-standard   VMware, Inc.  PartnerSupported
In the HP Custom image there is only one profile, but you may see many options here, unless you know otherwise just use the -standard one. Now we know the name of the profile we can update the host. I recommend doing a dry run first.
~ #  esxcli software profile update --depot=/vmfs/volumes/datastore1/patch/update-from-esxi6.5.zip --dry-run --profile=ESXi-6.5.0-20171103001-standard

Update Result
   Message: Dryrun only, host not changed. The following installers will be applied: [BootBankInstaller]
   Reboot Required: true
   VIBs Installed:...
   VIBs Removed...
Once you are happy you can run it without the dry run option.
~ #  esxcli software profile update --depot=/vmfs/volumes/datastore1/patch/update-from-esxi6.5.zip  --profile=ESXi-6.5.0-20171103001-standard

Update Result
   Message: Dryrun only, host not changed. The following installers will be applied: [BootBankInstaller]
   Reboot Required: true
   VIBs Installed:...
   VIBs Removed...
Once is complete you need to reboot the host.
~ # reboot
When the box is rebooted run the following, to check the new version has applied.
~ # esxcli system version get
    Product: VMware ESXi
    Version: 6.5.0
    Build: Releasebuild-5310538
    Update: 0
    Patch: 19
If the update fails, simply reboot the server and ESXi will roll back.

Hyper-V & OVH – Workgroup

Hyper-V & OVH – Workgroup

On Hyper-V Server

  • Enable Remote Management
  • Enable Remote Desktop
  • Set Computername
  • Enable WSMan and PS-Remoting:
Enable-PSRemoting  
Enable-WSManCredSSP -Role server  

On Managing Client

  • Configure hosts or DNS entry for Hyper-V Server
  • Ensure network profile is “Private”:
Set-NetConnectionProfile -InterfaceAlias Ethernet -NetworkCategory Private  
  • Temporarily start WinRM and add Hyper-V Server as Trusted Host:
Start-Service -Name winrm  
Set-Item WSMan:\localhost\Client\TrustedHosts -Value "hyperv-server.home.local"  
Stop-Service -Name winrm  
  • Enable Hyper-V Management Tools in Windows Features
  • Add Hyper-V Administrator credential with cmdkey:
cmdkey /add:HYPERV-SERVER /user:Administrator /pass:  
  • Launch Hyper-V Manager and connect to server (don’t specify user)
    • or, launch Hyper-V Manager with runas:
runas /user:HYPERV-SERVER\Administrator /netonly "mmc virtmgmt.msc"  

Summary

After lots of mucking around, this is the bare-minimum working configuration I’ve found to remotely manage a Hyper-V server in a non-domain environment.

I’m still not 100% certain why Hyper-V Manager does not work with supplied credentials and the cmdkey or runas workaround is necessary.

Stolen from: https://blog.ropnop.com/remotely-managing-hyper-v-in-a-workgroup-environment

Hyper-V & OVH – Network

Hyper-V & OVH – Network

Network (powershell script)

 

Create an External Switch from the command line

This is the hard part that really had me banging my head of the wall.  If you create an external switch you lose connectivity.

You need to determine the Adapter name to use.  Do to this from a Power Shell prompt run:


Get-NetAdapter

This will list the name of your adapters.  Figure out which one you want to use for your switch.
In my case it was Ethernet 2.

Save the Power Shell Script below to a ps1 file (say c:\MakeSwitch.ps1).  You can run:
Notepad c:\makeswitch.ps1
To do this.  Update it with the correct name determined above.

<#
Make Switch for So You Start Dedicated Server
Usaage:
execute the command Get-NetAdapter
This lists the ethernet adatpers.
Replace “ethernet 2” below with the name of your connected physical adapter
Adapted from:
http://blogs.technet.com/b/heyscriptingguy/archive/2013/10/09/use-powershell-to-create-virtual-switches.aspx
#>

Import-Module Hyper-V
$ethernet = Get-NetAdapter -Name “ethernet 2”
New-VMSwitch -Name externalSwitch -NetAdapterName $ethernet.Name -AllowManagementOS $true -Notes ‘Parent OS, VMs, LAN’
New-VMSwitch -Name privateSwitch -SwitchType Private -Notes ‘Internal VMs only’
New-VMSwitch -Name internalSwitch -SwitchType Internal -Notes ‘Parent OS, and internal VMs’

This actually creates three switches, one of each type.   (external, internal and private).