vSphere Big Data Extensions v1.0 Beta

VMware vSphere Big Data Extensions 1.0 Beta - Release Notes

VMware is proud to announce the VMware vSphere Big Data Extensions v1.0 beta. This product is designed to work with vSphere and leverages the VMware contributions to the Serengeti Project.

Big Data Extensions gives customers an easy to use management tool to provision, manage, and monitor enterprise Hadoop clusters on vSphere through the vCenter user interface.

What's New in vSphere Big Data Extensions

vSphere Big Data Extensions is a virtualization platform that enables provisioning and lifecycle management of Hadoop on VMware vSphere. This release provides the following features andProject Serengeti enhancements.

  • Big Data Extensions Graphical User Interface.

    You can perform the following tasks with the Big Data Extensions graphical user interface.

    • Create, scale-out, and delete clusters.
    • Manage vSphere resources for use by Hadoop clusters.
    • Manage and monitor Hadoop clusters.
    • Control Hadoop resource usage.
  • Support for all Major Hadoop Distributions.
  • Automatic Elasticity.
  • Adjust the Compute and Memory Resources of Running Hadoop Clusters.
  • Disk Failure Recovery.
  • Create a Hadoop Virtual Machine with a Custom Linux CentOS Operating System Configuration.
  • Support for vSphere Standard Edition When Using the Technology Preview.

vSphere Big Data Extensions Administrator's and User's Guide
vSphere Big Data Extensions Command-line Interface Guide

I am sure this will be of use to those who are already using Project Serengeti


All I Did Was Add a VMkernel Interface (Routing)

That was the call I got today.
"All I did was add a VMkernel interface and my host lost connection to vCenter".

On went my troubleshooting hat.

First the environment (simplified)


The physical interfaces on which the VMkernel interfaces reside were trunked with multiple VLANs. In this case VLAN(4) and VLAN(49).

vmk0 was used for ESXi management - with a default gateway of x.x.4.254

When the user added vmk1 - the host would become disconnected, he removed vmk1 - the host reconnected to the vCenter.

While the host was disconnected, we tried to ping the vmk0 interface - replies were fine.

While the host was disconnected - we tried to connect to the host directly with SSH and the vSphere client - both worked.

While the host was disconnected - we tried to ping the vCenter server with its IP - there was no response.

While the host was disconnected - we tried to ping the external network - replies were fine.

We then looked at the settings on vmk1 - and I noticed that the user had not set the VLAN49 tag on the VMkernel interface. Obviously this was not set correctly, and by adding VLAN(49) to vmk1 - everything worked correctly. The Host reconnected to vCenter.

So the problem was solved - VLAN(49) was missing on vmk1. ????

I was puzzled and tried to understand why this misconfiguration would cause the host to disconnect from the vCenter - then I realized why, and therefore the reason for this post.

When configuring a VMkernel interface, a new entry is added to the routing table. There will only be one default gateway - and that will the one defined on the Management interface. The additional VMkernel interfaces will not have a gateway defined.

This was the printout of the esxcfg-route -l from the Host.


Just to explain the output in plain English.

Anything on Network x.x.4.0 on that subnet will go out through vmk0.
Anything on Network x.x.6.0 on that subnet will go out through vmk2.
Anything on Network x.x.49.0 on that subnet will go out through vmk1.
Everything that does not match the above - will go out the Default Gateway x.x.4.254 through vmk0.

So the user had configured vmk1 on x.x.49.0. That meant any traffic trying to go out to the vCenter server - would go out through vmk1 - it was on the same subnet.

But… the user had not assigned the appropriate VLAN(49) tag to the interface - which meant that the interface would send out packets onto the network but without the correct VLAN tag on the packets, and therefore the Host could not communicate with the vCenter.

Photo by Jelle Oostrom (flickr)

Always follow the route..


Security in VMware Virtual Appliances

Today I got a reminder of a post I have been meaning to write about security best practices and VMware Virtual Appliances.

A question was raised on the VMTN community VCSA - what is the default "upgrade" user for?

VMTN question

That is a very legitimate question!! Giving a user all rights with no password can become an issue. But in this case since it is used (most probably) for the purpose of the upgrade of the VCSA from version to version - then it might OK (or not… ).

Which leads straight into the next subject - what users (and what are their rights) exist on the vCenter Server Appliance?

Let's have a quick look at what users there are on the VCSA (screenshot is all the local users on my VCSA that are both not locked or disabled)

locked+disabled users

So firstly let me say that is a hell of a lot of users in the server that in essence is one of (if not the most) important part of your Virtual infrastructure.

You might say (and rightfully so) that even if the users are defined - that does not mean that they can actually log into the system - for that they would need some kind of shell access. So I checked which users actually have shell access. That would be users who do not have /bin/false or /sbin/nologin in their profile. This is what I got

shell enabled users

Let's concentrate on some of the users in the list.

lp Access to printer hardware; enables the user to manage print jobs.
ftp FTP user
man Used for man
games Access to some game software
news Used for news application
uucp Serial and USB devices such as modems, handhelds, RS-232/serial ports.

For someone that values security - all our Linux boxes are hardened (as they should be), so the first thing we do is run the following:

# userdel shutdown
# userdel halt
# userdel reboot
# userdel games
# userdel news
# userdel gopher
# userdel ftp

(Just to clarify - I do not advise running any of the above commands on your vCenter Server without proper testing and approval from VMware and doing so might void your support.
Be warned!!)

If you were to ask me - most of the users listed above have absolutely no business being on a production box… especially not on my vCenter!!!!

In VMware's defense I should say that I checked which users were available on other VMware appliances - such as the vCOPS appliances and the new Log Insight appliance - and most of these users were not present on either of them. Perhaps this is the way going forward.

VMware have made progress - but I still do not feel 100% comfortable with the vCenter virtual appliance even after I wrote my post last year - Should You Patch the vCenter Server Virtual Appliance? 

vCenter has all the keys to the kingdom - and VMware must make the utmost effort to make sure that no possibility of exploit can used by leaving silly holes and possible security vulnerabilities open in the underlying operating system.

The same way that a ESXi host is locked down - there is no reason why the vCenter server should users like games on them.

ESXi Users

Imagine if we had Pong on our vCenter…

Game of Pong?


Yo Ho Ho - VMware Needs a CTO…

Forgive the catchy title…

Well this morning I thought that I had missed something (which I usually don't) - and that Paul Strong was appointed as the new CTO succeeding Stephen Herrod.

What made me come to that conclusion you might ask - it was this article.


Performing a reality check on Twitter is easy

And it seemed I was not the only one.


I have heard from within VMware that Paul Strong was not appointed as the new CTO (he is the CTO for Global Field- but not THE CTO), so I do not know where The Register got their information fromimage.

Which now brings me to another and more important question. How long can VMware go on without filling Stephen's shoes? Stephen left his position as the CTO on January 15th, 2013.


Shortly  thereafter (related or not I do not know - I am not a financial buff) the VMW stock took a beating, dropping almost 20 points, today the stock it still at around the same level (after some ups and downs).

Stephen was (and still is) highly respected for his leadership qualities, his vision and his part in bringing VMware to the place they are today. I do not think there were many who were not shocked about the announcement - it was not something that many predicted. The community - especially the active virtual community hold him in high regard.

So the obvious question is what is taking so long to replace him? I do say that his replacement will have to live up to high expectations, from the shareholders, from the community and also from the customers. His spot is not an easy one to fill. I personally do not know enough about the "visionaries" within VMware who can replace him, or perhaps it will be someone from within the "mothership" (gotta love speculation).

I do think that VMware have to fill this void - the sooner the better. It has been 5 months. It is not good for business, it is not good for your stock.

It will restore peace to the the world, calm to the schizophrenics and ..

Disturbance in the force

Please feel free to leave your comments and thoughts below.


Announcements from Red Hat summit

Here is a short summary with links on today's announcements from the Red Hat summit in Boston.

Red Hat Launches Red Hat Enterprise Virtualization 3.2

Red Hat Enterprise Virtualization 3.2 brings a vast array of new features, including:
  • Fully supported Storage Live Migration, allowing virtual machine images to be moved fromimage one storage domain to another without disrupting service
  • Support for the latest industry-standard processors from Intel and AMD, including Intel Haswell series and AMD Opteron G5 processors
  • Enhancements in storage management, networking management, fencing and power management, Spice console enhancements, logging and monitoring, and more.

New Third-Party Plug-ins
Red Hat is already collaborating with several industry leaders to integrate their solutions with Red Hat Enterprise Virtualization via the new plug-in, including high availability and disaster recovery solutions from NetApp (with a VSC), Symantec (HA), and Insight Control from HP

Red Hat Shows OpenStack Ecosystem Strength; Launches Red Hat OpenStack Certification, Unveils Red Hat Certified Solution Marketplace

As part of today’s announcement, IBM joins Cisco and Intel as Alliance Partners in the Red Hat OpenStack Cloud Infrastructure Partner Network.
Red Hat OpenStack Certification Program
Red Hat Certified Solution Marketplace

Red Hat and Mirantis Partner Across Products and Services to Accelerate Adoption of Red Hat OpenStack

Mirantis, and Red Hat today announced that the two companies will collaborate to optimize Mirantis’ Fuel tools for deployment of Red Hat OpenStack, and deliver OpenStack implementation and integration services to joint customers.

Red Hat Announces OpenStack-powered Product Offerings to Deliver on Open Hybrid Cloud Vision

New solutions include extension to Red Hat Enterprise Linux product family and new offering to enable customers on their journey from datacenter virtualization to Infrastructure-as-a-Service. Red Hat today announced two new product offerings with one vision of delivering an Open Hybrid Cloud. The new offerings include Red Hat Enterprise Linux OpenStack Platform, a solution that serves as the foundation for advanced cloud users who are seeking to build an OpenStack-powered cloud, and Red Hat Cloud Infrastructure, a comprehensive offering designed to support organizations on their journey from traditional datacenter virtualization to OpenStack-powered clouds.


vCenter Log Insight Now Available

Hello World….

vCenter Log Insight (Release Notes) - The first public release of the new Log Management and Analytics product.

VMware vCenter Log Insight is the new solution of VMware for log management and analytics for dynamic hybrid cloud environments. It delivers superior technology for automated log management through log analytics, aggregation, and search to extend the leadership of VMware in analytics to log data. Log InsightLog Insight can analyze vast amounts of unstructured machine generated data and enable interactive, real-time search and analytics through an easy to use interface providing superb time to value. It analyzes log data of all types and from all devices, enabling deep, enterprise-wide visibility. With a focus on integrated cloud operations management, and an analytics driven approach, Log Insight provides the operational intelligence needed to proactively enable service levels and operational efficiency in dynamic hybrid cloud environments.

Think of it as something similar to Splunk but different - it is specifically vSphere Centric (at least at the moment), built by VMware people (as a result of the Log Insight acquisition from August 2012) and it integrates with vCOPs (which is a great plus)

There is recommended sizing document



VMware vCenter Log Insight is available for purchase as a standalone product. It has a simple pricing model, with one flat rate for any server, virtual machine or vSphere host from which you collect logs.

VMware vCenter Log Insight is licensed on a per operating system instance (OSI) basis, which is defined as any server, virtual or physical, with an IP address that generates logs, including network devices and storage arrays.

With Log Insight, you can analyze an unlimited amount of log data per OSI. The advantage of this is a simple and predictable pricing model that is based on the size of the infrastructure; it does not force you to buy additional licenses to cover the worst-case scenario and pay more for increased log volumes.

Given that systems and devices can generate huge amounts of log data during peak times, or while monitoring and troubleshooting for various IT issues, this is an important distinction.

Detailed pricing information will be announced when vCenter Log Insight is ready to ship
in Q3 2013

Just as a side note… Version 1.0 should not have a version number of 0.9.1 - that does not make sense…

1.0 or 0.9.1
1.0 or 0.9.1

Give it a spin - and le me know what you think about it?


vExpert 2013

Last week, the 581 people that were awarded the vExpert title for the year 2013. It is large list of people who are active in the community, that share knowledge, that lead VMUG's and all other kinds of evangelizing for VMware and the community in general.

As we all know there are a number of "perks" that come with being a vExpert, but mostly it is an honor.

  • An honor to be part of an amazing group of people vExpert
  • An honor to serve the community
  • An honor that people acknowledge your contributions

I have traditionally created a Twitter list of the vExperts each year, and this in not different.

I actually created two of them due to the fact that Twitter limited the number of members of a single list to 500 members (and of course they changed it on Thursday - after the lists were populated)

So there will be only a single vExpert 2013 list which you can find here.

Thank you for the honor and here is looking forward to a wonderful and exciting year..