Configuring custom MTU size on vmxnet3

A few weeks ago I visited a customer who had a few servers configured with vmxnet2. I asked him why and he told me that if he changed nic to vmxnet3 he would loose the ability to configure a custom MTU size. With vmxnet2 the MTU size can be configured directly from the driver by specifying the size you want while on vmxnet3 you can only choose between the standard (1500) and Jumbo Frames (9000).

As this customer had ipsec communication between servers and GRE encrypted WAN links, none of these options would work too well. He needed to set a lower MTU than 1500 to keep things running.

While there exists some registry hack we found that the best method to configure a custom MTU was not in the driver (or registry), but rather by using the command line utility netsh. This solution is not vmxnet3 specific, but a general solution that should work independent on the type of nic you're using.

You will first need to figure out the index number of your nic with the command netsh interface ipv4 show interfaces
After finding the index number (16 in our case) you will need to use it in the next command (NB: needs admin privileges) netsh interface ipv4 set subinterface "16" mtu=1340 store=persistent


PS: The size of 1340 is just used as an example here. I don't remember the exact MTU used in this scenario, but I'm pretty sure it was 1300-something. If in doubt you can use tools like ping or mturoute to find the optimal size.

Creating Horizon View 5.2 pools with Powershell

Most people have used the View Manager GUI to create new pools in their VMware Horizon View environments. In order to access some of the new functionality you are however required to use PowerShell (View PowerCLI) instead. The syntax for creating a linked clone pool is a follows:

NAME
    Add-AutomaticLinkedClonePool
SYNOPSIS
    Adds an automatically provisioned linked-clone pool.

SYNTAX
Add-AutomaticLinkedClonePool -Pool_id [<string>] [-Description [<string>]][-DisplayName [<string>]] [-Disabled [<Boolean>]] -Vc_id [<string>] [-Persistence [<string>]] -VmFolderPath [<string>] -ResourcePoolPath [<string>] -ParentVmPath [<string>] -ParentSnapshotPath [<string>] -DatastoreSpecs [<string>] -Composer_ad_id [<string>] [-OrganizationalUnit [<string>]] [-LogoffScript [<string>]] [-PostSyncScript [<string>]] [-UseUserDataDisk [<Boolean>]] [-UseTempDisk [<Boolean>]] [-DataDiskLetter [<string>]] [-DataDiskSize [<string>]] [-TempDiskSize [<string>]] [-RefreshPolicyType [<string>]] [-RefreshPolicyDays [<string>]] [-RefreshPolicyUsage [<string>]] [-CustomizationSpecName [<string>]] [-MinimumCount [<string>]] [-MaximumCount [<string>]][-HeadroomCount [<string>]] [-MinProvisionedDesktops [<string>]] [-DeletePolicy [<string>]] [-DnsSuffix [<string>]] -NamePrefix [<string>] [-PowerPolicy [<string>]] [-IsProvisioningEnabled [<Boolean>]] [-SuspendProvisioningOnError [<Boolean>]] [-IsUserResetAllowed [<Boolean>]] [AutoLogoffTime [<string>]] [-AllowMultipleSessions [<Boolean>]] [-FolderId [<string>]] [-DefaultProtocol [<string>]] [-AllowProtocolOverride [<Boolean>]] [-FlashQuality [string>]] [-FlashThrottling [<string>]] [-UseSeSparseDiskFormat [<Boolean>]] [-SeSparseThreshold [<string>]] [-NetworkLabelConfigFile [<string>]] [<CommonParameters>]

DESCRIPTION


RELATED LINKS
REMARKS
    To see the examples, type: "get-help Add-AutomaticLinkedClonePool -examples".
    For more information, type: "get-help Add-AutomaticLinkedClonePool -detailed".
    For technical information, type: "get-help Add-AutomaticLinkedClonePool -full".

As you can tell, this command can take quite a few parameters. In our case we had to specify quite a few of them as well in order to get things configured correctly. Figuring out the required parameters is easiest if you have an existing pool and run the command Get-Pool -Pool_id Poolname

Since we have multiple pools on multiple vCenter servers we would also get the advantage of pools that would have the exact same config if we scripted the configuration as human error is always a risk. The feature that triggered our interest in using this command was the new networking functionality where a single pool could span multiple VLANs. The VMware documentation showed us exactly how to get the needed config file setup. 

While we were able to create pools that used multiple VLANs by using this command we discovered a couple of limitations:

1. We were not able to specify a user drive letter (got an error message if we specified it)
2. 3D settings are not available for powershell
3. Storage Accelerator settings were not available

We had to create manual steps for setting these options. We hope VMware will eventually fill this gap and make all options available from both powershell and the GUI.

Saving old tape recordings using ESXi and Ubuntu

Background
For years I've had piles of old Digital-8 (plus a few Hi-8) tapes stored that will eventually become very hard to read as the hardware supporting them has not been for sale in many years. The existing units that are still alive are getting old (I had this cam to the repair shop twice during the last two years of service (2000-2009)). Getting the tape data to a more accessible storage device is important or I'm risking it will be lost forever, stuck on media that nobody can read.

Solution

1. ESXi 5.1
2. VT-d enabled in BIOS on the host
3. Physical server with IEEE 1394 firewire card. I had one integrated on the motherboard (Gigabyte GA-990FXA-UD3, Socket-AM3+)
4. Host / Hardware / Advanced Settings / DirectPath IO / Edit / Select 1394 device
5. Ubuntu Server 10.04 LTS with dvgrab
6. sudo dvgrab -a -format raw -rewind -t prefix-

One drawback I had during this setup was that my two port Gig nic also had to be allocated for sharing. I didn't have to configure it for my particular VM, but I could no longer use it for vSwitching. :(

In the VM I added the firewire card under Add / PCI device.

lspci revealed that my VM could indeed see the firewire card.

The command I'm using to rewind a tape and capturing the data is:

sudo dvgrab -a -format raw -rewind -t prefix-t

I'm also using a script that copies the captured data to a file server after finishing the capture job.

The VM was  a standard install of Ubuntu Server 10.04 LTS and I've included a list of installed packages here.

When creating your VM, keep in mind that a 90 minute Digital-8 tape has about 19 Gigabytes of data.

Note:
I initially tried using Ubuntu Server 12.04 LTS, first in a VM, and then on a physical box. Both attempts failed even though it looks like at least one guy has gotten it working, but I found several others who had the same problems as me (Error: no camera exists). I did however find a posting indicating that this could be caused by a problem present in newer versions of Ubuntu.

References:
http://kb.vmware.com/kb/1010789
http://www.vmware.com/files/pdf/techpaper/vsp_4_vmdirectpath_host.pdf
http://software.intel.com/en-us/blogs/2009/02/24/step-by-step-guide-on-how-to-enable-vt-d-and-perform-direct-device-assignment

http://forum.ubuntuusers.de/topic/ubuntu-12-04-dvgrab-firewire-dv-camera-wird-ni/#post-5224722

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1045047

http://www.mail-archive.com/ubuntu-bugs@lists.ubuntu.com/msg3469365.html

http://sharpeespace.blogspot.no/2012/06/capturing-firewire-1394-camcorder.html

Adding local storage disconnects the ESXi 5.x host

Adding local disks may on some servers lead to the ESXi 5.x host to be disconnected from vCenter. If you're connected directly to the ESXi host you will be unable to connect to it again.

In the vmkernel log you will see messages about how it fails to add the disk and it ends with "dumping cartel".

In order to be able to access the server through the vSphere client again you will need to restart the server's management agents. 

This needs to be done from the local console of the ESXi host which is easiest available through DRAC (or similar). You can find the correct menu under Troubleshooting options.

After situations like these, restarting the agents may take a few minutes.

The reason that adding the local disk crashes is that this server was shipped with a preconfigured disk with several partitions. If we remove these partitions first, these problems will not occur.

From the ESXi's command line (ESXi Shell (troubleshooting mode)) of ESXi we can use partedUtil to achieve this.

There are normally multiple volumes presented to an ESXi host and we do not want to delete the wrong one(s). You may need to run partedUtil against several of your volume IDs before you hit the correct one. You can then delete partition by partition from that volume by using partedUtil delete.

Now you will be able to add the disk as expected.

Thanks to Marcos Silva who pointed me in the right direction for finding a solution to this problem.

Enabling FCoE on HP Servers

Newer servers come with CNA adapters preinstalled, but it is also possible to insert additional cards if needed. In order to use FCoE on a new server you will normally need to enable it from the BIOS (iSCSI is default).

In HP Servers the setup is pretty straight forward and you can enable it from the BIOS using the following steps:

1. Press F9 during POST to enter the BIOS
2. Select System Options
3. Select Nic Personality options
4. Press Enter on each of the nics and select FCOE
5. Exit & Save

Enabling FCoE on Dell servers

Newer servers come with CNA adapters preinstalled, but it is also possible to insert additional cards if needed. In order to use FCoE on a new server you will normally need to enable it from the BIOS.

I found the setup from the Dell BIOS a bit strange, but it probably made complete sense to the developer(s) who designed this.

In ESXi 5.x you don't need any extra drivers in order to use FCoE.

The steps are as follows:

1. During POST press F2 to enter System Setup.
2. Choose Device Setup
3. Choose one of the NICs
4. Choose NIC Partitioning Configuration
5. Choose Partition 4
6. Set FCoE Offload Mode to enabled
7. Save
8. Repeat steps 3-7 for each NIC in the server

A big thank you to @SolTecLTD who pointed me in the right direction.

Stretching data centers through hacks and kludges

Earlier this week my company had a workshop at Bella Sky in Copenhagen. We have such gathering twice a year, TechnoCamp, where a group of consultants meet to discuss and develop different solutions. This time was the 11th time such a gathering was held.  #technocamp  #tc11

The main focus this time was virtualization, disaster recovery and HA. I was part of a group discussing stretched data center networks.

Stretching data centers is becoming more and more popular (RTO && RPO=0) and is a requirement in order to support long distance vMotion. But as we all know, stretching layer 2 has a few drawbacks that need to be taken care of.

On the storage side, the split brain issue needs to be fixed and may cause serious headaches if it's not.

On the networking side we have other challenges that may bring down the network on both sides unless implemented correctly.

You may have heard about OTV and think that it is a silver bullet for such solutions. Well... I guess it's not quite that straight forward.

I taped (disked?) an interview with (soon to become) CCIE Håvard Nyhus (Twitter: @hnyhus Blog: http://technet.gathering.org/) where we are discussing these issues. The spoken language is Norwegian with English subtitles.

Since subtitles isn't working in the inline player, you should probably watch it directly on YouTube: http://youtu.be/ajD_UJhVCW8