iLO 4 causes PSOD

A majority of the environments I work on are HP, Cisco UCS, or Dell servers these days. Not in any particular order, but that just seems to be what I see the most of. I have been seeing a lot of activity related to an issue I have not personally run into but HP has released a tech bulletin on. That issue is related to HP's iLO 4 firmware. The issue seems to be the following;

"HP iLO 4 may experience Intermittent Non-Maskable Interrupt (NMI) Events on Proliant G8 Servers with HP iLO 4 firmware versions 1.30, 1.32, 1.40 and 1.50."

If you experience these NMI events it is possible for bad things to happen on your OS

• VMware ESXi hosts may experience Purple Screen of Death (PSOD)
• Linux Operating Systems will display a Message indicating an NMI occurred
• Microsoft Windows will experience a Blue Screen of Death (BSOD)

HP's Tech Bulletin concerning this issue can be located here

HP's solution to this issue is to update to iLO 4 version 1.51 or later, 1.51 can be found here

VTUG New England Summer Slammer 2014!!

Big thanks to everyone for coming to the Focus booth at VTUG yesterday. Also a special thanks to those who sat in on my Troubleshooting tips for View breakout session.

-Brad

Heartbleed issues in virtual environments

I've had a lot of customers email in asking about openSSL heartbleed related concerns after the announcement last week. There is an informational website setup here to explain the issue in detail.

First thing I want to say here is that this is information that  I've been able to track down, but in no way is this a complete list of everything effected. It is merely a list of items I currently work with that happen to be a part of a lot of my customers solutions. This issue is widespread and evolving quickly so there will obviously be changes to what is listed in this article.

OpenSSL released a security advisory on this issue;

"OpenSSL Security Advisory [07 Apr 2014]
========================================

TLS heartbeat read overrun (CVE-2014-0160)
==========================================

A missing bounds check in the handling of the TLS heartbeat extension can be
used to reveal up to 64k of memory to a connected client or server.

Only 1.0.1 and 1.0.2-beta releases of OpenSSL are affected including
1.0.1f and 1.0.2-beta1.

Thanks for Neel Mehta of Google Security for discovering this bug and to
Adam Langley <agl@chromium.org> and Bodo Moeller <bmoeller@acm.org> for
preparing the fix.

Affected users should upgrade to OpenSSL 1.0.1g. Users unable to immediately
upgrade can alternatively recompile OpenSSL with -DOPENSSL_NO_HEARTBEATS.

1.0.2 will be fixed in 1.0.2-beta2."

Basically this issue allows somebody to grab 64k chunks of data out of memory on a server utilizing OpenSSL. As a result this data could be used to figure out the private key associated with certificates used to secure content. The result of this would be the private key being used to de-crypt a datastream and view what is supposed to be a secure encrypted transmission between two endpoints. Most of the manufacturers are scrambling to figure out a solution to this issue. I will provide some info and useful links to the Vendors we commonly work with to supply our customers with top of the line solutions.

VMware

VMware has released information on this issue including which products are affected by the security vulnerability. This official VMware blog post has some info, however you'll want to view this KB article to see the products and info affected.

Products Affected include

EMC

EMC has posted an advisory here you'll need powerlink/support credentials to get in to view the advisory. The vast majority of the products are NOT affected. The list of effected products is

Cisco

Cisco has also released a statement on this issue as well. They have a preliminary list of devices that are affected, seen below, but follow the link in above to get to the statement. The list will likely changed, they have gone through the entire portfolio yet. Most notably the Cisco UCS platform seems to be in the clear.

Hewlett-Packard

HP also released a statement, not with much detail, however they have ruled out some of the product line. The statement can be found here.

Teradici

Anyone using VMware View and Zero Clients should note that the PCoIP Management Consoles from version 1.9.0 to 1.10.0 are effected. An upgrade will fix this. More info here

Microsoft

Last, but not least this time is Microsoft. Not surprisingly Microsoft products seem to be uneffected because they don't use openssl typically for anything. IIS, among other secured products in their portfolio, do not use OpenSSL and is therefore uneffected.


In closing, I would recommend following the advice of the manufacturer in resolving the issue. Also if you have management devices that are affected that are on a private VLAN don't worry as much about them because you have physical control over who's accessing them. Start with your most public facing devices and work your way back into the network.

VMware View 5.3 with NVIDIA Grid Technologies with vSGA

I recently had the experience of working with the NVIDIA Grid technologies within VMware View 5.3 using the, now production, vSGA technologies. vSGA is basically the ability to share GPU's and VRAM (I know everyone hates this term from the licensing debacle VMware had, but now VRAM means video RAM, forget about the old use of the term). It took a bit of research to figure out the basic steps to make the Grid cards useful within the ESXi host.

Surprisingly enabling the 3D graphics support was really easy. There's an option on the VM to enable 3D graphics support, then you need to enable it on the View Desktop pool that you want to use it with. Before any of this gets done we have to prep the ESXi host to be able to use the adapter. You can take a read through the deployment guide to get all the details but I'll give you the quick and dirty version;

First we need to build our host and get the driver loaded, the driver can be downloaded from NVIDIA

1. Install ESXi 5.5
2. Put host into maintenance mode
3. Load the NVIDIA Grid VIB
1. esxcli software vib install –-maintenance-mode –d /vmfs/volumes/VNX_SAS_ISO/NVIDIA-VMware-x86_64-304.59-bundle.zip” (replace file path if necessary)
4. Reboot host
5. SSH to host
6. Check to see if the “Xorg” service started

Next Check to see if the Driver associates with the correct card

Next we want to check that the GPU VM sees the card and is loaded and managing the card

This process has to be done for each host and basically at this point you can deploy your parent VM (be sure to take your snapshot and that the View agent is loaded). Enable the 3D graphics on the virtual video card of the VM and enable the 3D graphics on you desktop pool. After all that is done you can check to see if utilization goes up and down on the GPU's in the card. You should see the volatile GPU-Util % should fluctuate as GPU is needed.

I've found this makes a significant improvement on graphics performance for a view session.

Hope this is helpful

-Brad