January 26, 2015

Bulk registering vSAN disks for controllers not supporting pass-through mode

When configuring VSAN the amount of initial setup time is highly dependent on the type of disk controller you're using. Some controllers support pass-through mode and will not need the additional configuration described in this posting.

If you however are using a controller such as the Dell PERC H710, you will first need to setup each disk in the RAID controller's BIOS; with every disk in it's own disk group where you enable write through, disable read ahead and select initialize.

After doing this you will see the individual disks within VMware vCenter under the esx host / manage / storage / storage controller / devices. The disks are however not detected correctly as the controller gives no information about the type of disks shared in these RAID 0s.

In order for vSAN to make sense of these disks you will need to create rules that specify what type of disks that are being used.

Spinning disk command:
esxcli storage nmp satp rule add --satp=VMW_SATP_LOCAL --device <device id> --option "enable_local"

SSD disk command: 
esxcli storage nmp satp rule add --satp=VMW_SATP_LOCAL --device <device id> --option "enable_local enable_ssd"

The device id in question here is the naa lun id. Some suggest that you use the command esxcli storage core device list, but in a system with many disks I've found it easier to filter out the needed info by using the command fdisk -l by identifying the disk types by looking at the disk sizes.

You can compile the list of naa lun ids for a given disk type and run the following commands:
for i in <paste list of spinning disk naa lun ids here>
esxcli storage nmp satp rule add --satp=VMW_SATP_LOCAL --device $i --option "enable_local"

for i in <paste list of ssd disk naa lun ids here>
esxcli storage nmp satp rule add --satp=VMW_SATP_LOCAL --device $i --option "enable_local enable_ssd"

You will now need to reboot the host for the new config to become active. Repeat these steps for all of your vSAN hosts and you'll soon be able to start configuring vSAN.

November 22, 2014

vSAN and HP 5400 switches

While setting up vSAN we found several guides for Cisco switches, but none for HP. Even the HP vSAN reference architecture was using Cisco Nexus switches.

We did initially see the error message: "Host cannot communicate with all other nodes in the VSAN enabled cluster" even though all vSAN enabled vmkernel interfaces could ping each other. vSAN has some special multicast requirements that needs to be taken care of.

We were trying to get HP 5400 series 10GbE switches to work with vSAN.

After playing around for a bit with the switch config we came up with the following working config:
vlan 53
   name "vSAN network 1"
   tagged C1-C8
   ip address
   ip igmp
Within a few minutes the error messages were gone, status went to Normal with a green icon and vSAN started working nicely.

Since we had 2x 10GbE nics dedicated to vSAN we also setup a secondary vlan for vSAN and bound each of the vlans to different nics in order to get maximum performance.

November 18, 2014

Accessing the GK Cloud Labs from Linux

Last week I attended vSAN training in Stockholm. The requirements for attending this class was that you needed to bring your own laptop with RDP capabilities.
When attending the class I discovered that there were a few extra things into this requirement. According to the class manual it required you to install an ActiveX component in Internet Explorer in order to get this working.

As I'm a Linux user they did of course not provide any info on how to do it, but that's part of the game I guess. In case I couldn't figure things out I could always start a Windows VM from within VMware Workstation. They did however provide info for Apple Macintosh users. By reading through the Mac docs I found what was really going on behind the scenes. The RDP session required a proxy config and encryption.

The standard Ubuntu RDP client didn't provide support for an RDP proxy, but I found an alternate client, called FreeRDP that I installed by following this HowTo.

I could now the access the labs by using the info from the login info sheet we had been provided with the following command:
xfreerdp  /v:cloud.labs.globalknowledge.net /d:gklabs /u:Wxxxx-Studentx-x /p:PassWord /g:gw1.labs.globalknowledge.net /w:1920 /h:1080 -nego
The connection now worked perfectly, even though it spent some time setting up the initial connection. Looks like it was trying to verify the certificate, even with the -nego switch that is supposed to tell it to ignore the certificate. Well, it does in fact ignore it in the sense you're not warned about a self signed certificate, but it still waits for it to time out before starting the connection.

All in all the training was a great experience, giving a better insight into vSAN than the HOL lab.

August 23, 2014

Making the XtremIO GUI Simulator work under Linux

While attending XtremIO training this week there was a bit talk about a GUI simulator for XtremIO. While not as good as the real thing it can be a good thing for learning to know the GUI and maybe show customers/colleagues how to admin the XtremIO. While XtremIO was bought by EMC they still seem to operate outside of  EMC and their GUI is not integrated into UniSphere.

The GUI Simulator is available as for download and exists in two flavors: Mac and Windows.

I downloaded the Windows version and I initially planned to try to run it in Wine, but I discovered that it really was a java application so I just needed to extract the correct files and install the required version of java.

I use Ubuntu 13.04 and did the following steps:

Install java runtime 1.8:

$ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer
$ java -version

Install Wine from Software Center if you haven't already. We will be using Wine to unpack the files inside the .exe file by installing it into a Wine container.  Locate the XtremIO GUI Simulator exe file (which is an installer) and right click it.
Choose Open with Wine Windows Program Launcher.

Choose to install the application.

After a bit the install will finish and all the files are extracted
 You will need to make the Simulator.jar file executable.
$ cd .wine/drive_c/users/lars/Local\ Settings/Application\ Data/XtremIO\ GUI\ Simulator/app/
$ chmod +x Simulator.jar

Navigate to the app folder using the file browser
Right click Simulator.jar and choose Open with Oracle Java 8 Runtime

Pick your choice, any choice.

Login with default credentials

And you're free to use the XtremIO GUI Simulator.
Note that the while the GUI Simulator is good for training it is not 100% equal to the real XtremIO GUI as the simulator seems to have a few bugs that are not present in the real GUI. It still gives a fairly good idea of how things work.

The GUI Simulator requires quite a bit of resources in order to run well so a slow PC without too much free ram will not be working greatly.

August 17, 2014

Lenovo losing it's Thinkpad roots?

When IBM sold off it's desktop line of products to the Chinese company Lenovo in 2005 many people thought that this would be the end of an amazing product line. After Lenovo took over we observed the opposite, things were actually getting better than before.

For many years I've been a happy die-hard Thinkpad user. My previous laptop was a T520. Before that I had a T500, T61 and T60. Thinkpads have traditionally been "built like a tank" and not changed much in physical build between different models. This has made the transition to a newer model totally safe, because you always knew what to expect.

Now that my T520 was getting old it was time to get a new one. My employer now has some sort of BYOD system (Bring/Buy Your Own Device) where you can choose between a range of products. You can choose to get a free one or you can pay some extra to get top models. I could have gotten a T540 for free, but chose to go for the ("better bells and whistles") W540 instead. The T and W series laptops are usually quite similar, but the W series are equipped with better GPU and larger SSD.

Such an upgrade would give me a computer that was similar to the one I had, but with new and better components. This was something I had done many times before so I didn't waste time on reading reviews since I had a good idea of what to expect.

The day the new laptop arrived I was not late installing my favorite desktop OS instead of the preinstalled Windows 8 that was default.

My disappointment was however endless as I figured out the new computer was unusable due to the way they have changed the keyboard/trackpoint layout; no "mouse" buttons and included an oversized touchpad left of the center of the keyboard. Instead of the buttons you are supposed to use push on the touchpad as if it had buttons. They have also included a numeric keyboard, reduced the number of rows and removed special keys for wifi, sound controls, mute, and removed leds for caps lock, num lock and lid light.

You see, I'm one of those guys who are not using an external mouse. I'm using the little red joystick in the middle of the keyboard that Lenovo refers to as Trackpoint. The little red stick was still there, but without those three buttons it was useless.

My anger and frustration was similar to the reaction of Hitler in this YouTube video:

Many years ago I used mouse as my main pointing device (like most desktop users), but I started getting mouse arm/elbow symptoms. I decided to try change my habits and start using that little pointing stick in the middle of the keyboard. My mouse arm started to recover and I also discovered that I would do things more efficiently as I didn't have to move my arm away from the keyboard in order to move the pointer.

I did some attempts on using the TouchPad of the new W540, but basic tasks, such as marking a text that was more than one page was giving me headaches. Video and picture editing was frustratingly hard, and you could just forget gaming. I started looking for alternate ways of solving this, and in the end I bought a Lenovo usb keyboard that had TrackPoint, buttons and it even lacked a TouchPad (I always disable the TouchPad).

I'm now using the W540 as my main computer and bring it everywhere. It works quite nicely now that I'm having a proper keyboard/pointing device setup, even though it shouldn't have been necessary.

Other than that it seems that the overall quality of the new Lenovo series is not as good as the  good old ones:

  1. The lid is thinner than before and lacks a grip for carrying.  You're probably better off closing the lid before carrying it.
  2. The lid has no lock mechanism.
  3. Why has the power connector suddenly become square and incompatible with all old adapters?
  4. Hissing sound! There's a hissing sound both from the speaker and when using a headset. A noise canceling headset solves this, but should not be needed for daily use.

The screen is however superb. At first I thought having a screen resolution of 2880x1620 on a 15.6" screen would be a bit too much. And for some applications it is, but in most situations it's awesome. It could be a good idea to adjust the DPI settings for your display manager. It gives you a very large work space and allows for more information on less space. I have also tested it outside in the sun thanks to it's IPS LED technology it's possible to work outdoors. It's not perfect, but better than my previous laptops and much better than a glossy thing that many vendors are selling.

The NVidia GPU is also very nice with it's 576 cuda cores, but it gets very hot when under high load and not suited for lap operations. By using an IR Thermometer I have recorded temperatures above 50C at two areas under the laptop (probably where cpu and gpu are placed).

I wish Lenovo would reconsider their design and bring back the good old buttons and also consider not to try to become Apple like Dell, HP and a few others seem to be trying to. Better stand out from the crowd with proven solutions.

If a Thinkpad is not a real Thinkpad anymore then there's no reason I should choose Thinkpad (the workaround with that Thinkpad usb keyboard+trackpad would work with any vendor and I also have to use it with my MIIX2 11 that also suffers from the same problems as the W540 except that it lacks the TrackPoint completely).                                                                                                                                                            

April 21, 2014

Tempdb database continues to grow after upgrading vCenter

After upgrading a small customer from 5.1 (Simple Install) to 5.5 we discovered that the system disk was filling up. This system was installed using the MSSQL Express version that ships with vCenter that by default will put databases on the system disk. We soon figured out that tempdb was growing quickly and we were worried that we would soon run out of disk space. When the SQL Server service (+vcenter) is restarted tempdb is cleared, but it would start growing quickly again if the services were started.

In this process we saw the file C:\Program Files\Microsoft SQL Server\MSSQL10_50.VIM_SQLEXP\MSSQL\DATA\tempdb.mdf  growing to 40 Gigabytes before we had to restart services to avoid running out of disk space.

We first suspected that the read_committed_snapshot option was enabled, but it turned out it wasn't:

We also changed the tempdb file location so it wouldn't fill up the data disk, but we were still worried about it's growth rate.

In the end we set a growth limit on the database from SQL Server Configuration.

We defined a fixed growth limit of 2000MB.

After doing this the server was running happily ever after and the disks wouldn't fill up. Setting a growth limit on tempdb may affect database performance negatively, but we found it more important that we wouldn't run out of disk space.

I later also discussed this issues with a database admin and he had also seen tempdb temporarily growing quickly after upgrades to different systems, and he said it would normally stop growing at some point and it's size would eventually go back to normal again.

March 27, 2014

Sphere 5.5 client cannot be installed on a domain controller

When trying to install the fat native vsphere 5.5 windows client  you get the message:
vSphere Client requires Windows XP SP2 or later. 
vSphere Client cannot be installed on a Domain Controller.
The vSphere client has always been possible to install on a domain controller. vCenter also used to be able to coexist on a domain controller, but after 4.0 was introduced it was no longer possible as itincluded a separate ADAM database.

Even though the fat native windows client is being phased out, you still need to use both the web client and the fat native client in order to access all the functionality. For standalone hosts, the native client is the only choice. 

Microsoft in general recommend that you avoid installing third party software on Domain Controllers and that seems to be the reason why VMware has included this check. According to this posting VMware's reason for doing this was:
We did this deliberately to enforce a Microsoft standard that our guys agree with - don't install software on a DC, but they made that decision in isolation. Nothing more than that.  So use the workaround safely and hopefully we can undo this in the future.
The workaround is to install the vsphere client from the command line using the parameter /VSKIP_OS_CHECKS="1".

Another workaround would be to use a thinapped vsphere client package instead. Hopefully we will see a 5.5 client someday at the Thinapped vSphere Client Fling page.