Migrating a Desktop VM to an ESXi Server

I have a Windows 7 VM running on my primary Mac notebook that I use mostly for one single application on a regular basis, and for occasional browser testing or other random tasks on rare occasions.  I have an ESXi server running on an old desktop machine, and I thought that it would be nice to move that VM to the server and just remote desktop to it.  It would have the added benefit of being able to run the app from one of my other machines.  I learned a few things along the way.

First, the straightforward thing to do would be to create a new virtual machine on the server, install Windows 7, and then install the application.  But like any good lazy programmer, I figured there had to be a better way.  What I _really_ wanted to do was just move my existing image off of my laptop onto the server.  Turns out it’s not very difficult, but there are a few steps and at least one “gotcha”.

First, if you don’t already have it, you need to download the OVF tool from VMWare.  Extract the .tar.gz file and open the *.pkg file to install it.  Then, at the terminal, run a command similar to this:

/Applications/VMware\ OVF\ Tool/ovftool ./Documents/Virtual\ Machines.localized/Windows\ 7.vmwarevm/Windows\ 7.vmx /Documents/ConvertedVM/

This assumes your current working directory is in your home directory, your VM’s are stored in the same place mine are, etc.  ConvertedVM is the directory where it will output the converted VM as an OVF.  Season this recipe to taste.  Now, you can deploy the OVF to your server in the usual way.

As I mentioned before, there is one “gotcha”.  I had recently upgraded to the latest version of VMWare Fusion, and allowed it to upgrade my virtual machine’s “hardware”.  Don’t do that.  As the link describes, once you upgrade to hardware version “10” you can no longer edit the hardware settings of your VM using the Windows vSphere client.  You can edit it with the “web vSphere client”, but apparently that is something that is not available for the free version of ESXi I’m running in my home “lab”.  I needed to change the hardware setting though, because in the migration I lost my network adapter and the Windows application in question needs network access.

I was able to work around it in a fairly brute-force way.  I removed the VM from the ESXi inventory (but did NOT delete the disk), re-created a new Windows VM and reused the existing disk image.  It worked fine, although I did have to reactivate Windows.  In the previous link there is also a workaround mentioned using “PowerCLI”, but I haven’t used that so can’t vouch for the solution.

EXCLUSIVE LIMITED-TIME BONUS TIP!  Gosh, I sure do miss the <blink> tag.  But, I digress.  I’ve been using “Remote Desktop Connection” to access Windows VM’s.  I believe it was installed as part of Office for Mac 2011, or maybe it was something I downloaded from Microsoft many moons ago.  If you’re still using that too, I’ve found that there’s actually a much better free remote desktop application (also from Microsoft) available on the Mac App Store.  Get it now while supplies last!

Tracking Down Bandwidth Hogs: Netflow Home Edition

Recently Comcast announced they were going to be doing some testing of data caps in certain markets, including mine.  I’m a cord-cutter and I’ve been very happy with Comcast’s speed and reliability, but when you view a lot of streaming video (Netflix, Hulu, Amazon Prime, iTunes)  and do a lot of cloud backups of your data, 300GB suddenly doesn’t seem so much.

Fortunately, Comcast has graciously decided to give its customers a grace period before they actually start charging for overages, so I’ve been trying to manage my data use a little more proactively.  Looking at previous months, I saw that I was typically using 500-600GB of data.  After cutting my streaming usage way back, it seemed that I was still using a tremendous amount of data and exceeding my cap well before the end of the month, and I couldn’t really figure out why.

It so happens that the company I work for makes a great enterprise product for finding just these sort of things, using NetFlow.  Unfortunately, we don’t (currently) produce a home edition, so I decide to try the next best thing.

The first step was to configure my router, which is running  DD-WRT firmware, to export “RFlow”.  As best I can tell, RFlow is an implementation of some version of NetFlow.  Presumably NetFlow is trademarked, so they have to call it something else.   Add to the mix nprobe and ntopng, and I was able to find that one host was using a very large percentage of the total bandwidth.



So the host, which is my Macbook Pro, had used over 1GB of bandwidth during the reporting period, but what really surprised me was that it wasn’t predominantly downloading data, but sending data.  This set off all kinds of alarm bells and gave me a mild panic.  I’m thinking virus, botnet, who knows what evil malware I may have gotten, despite considering myself fairly savvy.

Unfortunately, I couldn’t tell much more about the traffic other than it was SSL.  But now that I had it narrowed down to a host, I remembered a handy utility I’d discovered awhile back called Little Snitch.  It’s basically a firewall which allows you to selectively allow and deny connections from your Mac.  Being required to whitelist each application that requests access to the network gets pretty tiresome after awhile, so I had stopped using it.  But turns out, the latest version has a “passive” mode, where it will monitor what’s going on but won’t actively block anything.  I let it run for awhile and was able to collect some interesting data.


Outlook?  What the hell are you doing?  After about an hour it had sent nearly half a gigabyte of data.  I’ll grant you that my emails may be overly wordy at time, but I hadn’t actually sent any.

Turns out, Outlook has a rather nasty little bug having to do with folder syncing that makes it use a metric crap-ton of bandwidth.  There doesn’t seem to be a fix that I could find, so for now, I’m just shutting down Outlook when I’m not actively using it.

And thus ends my bandwidth hog detective story.

OSX Lion: ssh_askpass: exec(/usr/libexec/ssh-askpass): No such file or directory

I keep running into this.  For various reasons I need to use password-based authentication on some test boxes that are regularly rebuilt which makes using key based authentication difficult.

Ordinarily I set up an alias like this:

alias ssh1="sshpass -p secretpassword ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"

… which lets me use a command like:

sshl eric@devbox

But sometimes (and I swear, it doesn’t always happen, but I haven’t figured out the variable yet), it fails, and I get:

ssh_askpass: exec(/usr/libexec/ssh-askpass): No such file or directory

… and indeed, the ssh-askpass binary doesn’t exist.  After some flailing around and googling, I finally came up with this ugly hack.  I create a shell script with the following contents:

echo "secretpassword"

Then, I set the following environment variable, which directs sshpass to use my script instead of looking for the missing ssh-askpass

export SSH_ASKPASS=/Users/eric/scripts/ssh-askpass.sh

It’s ugly, but it works.  Of course, it assumes you’re always using the same “secretpassword”.  Some day I need to figure out a better way of addressing this, but today is not that day.

See also:  http://apple.stackexchange.com/questions/18238/mac-os-x-lion-and-sshpass

Unattended Installation (aka Silent Install) of Sun JDK in Debian

Posting this short note to my blog in the hopes it may help someone else, as I had a hard time tracking this info down.

Debian provides a package (sun-java6-jdk) for installing the JDK, but when you have a situation where you need to do an unattended installation (aka “silent install”) of the JDK you are stuck, because the install insists on making you interactively accept Sun’s licensing agreement. Passing the -y option to apt-get has no effect. I was able to find a link on Sun’s site for doing silent installs on Windows, but nothing for Linux.

Fortunately, I eventually tracked down, all you have to do is the following before you run your apt-get install command:

echo sun-java6-jdk shared/accepted-sun-dlj-v1-1 boolean true | debconf-set-selections

This indicates to the installer hooks that you have already accepted the license agreement, and doesn’t give you the annoying EULA dialog.

UPDATE 11/3/09: I should note that all of the above research was in preparation for setting up an automated install with FAI. Unfortunately, I later found out, that while the above works great in the regular command line, due to the nature of the environment FAI installs run in, it doesn’t work for that. However, I also discovered there is a correct way of doing it with FAI.

Use “debconf preseeding”. Create a class file e.g. debconf/DEFAULT (I used debconf/easberry-vm-std-plt-01 for testing with my host class, and populate it with the following:

sun-java6-bin   shared/accepted-sun-dlj-v1-1    boolean true
sun-java6-jdk   shared/accepted-sun-dlj-v1-1    boolean true
sun-java6-jre   shared/accepted-sun-dlj-v1-1    boolean true

This accomplishes basically the same thing, but in the FAI environment. Success!

Matching multiline regular expressions in Java

Regular expressions are one of those things that I understand but don’t use often enough to have really mastered. I use them even less in my Java programming. Today I found myself banging my head into my cubicle wall trying to parse out a file that looked something like this:
anotherthing=ran outta foo words
I needed to parse this out and create objects representing each section. Initially I started trying to read this in line by line, keeping track of where I was in relation to the markers, concatenating string buffers, etc. Then I realized how retarded that approach was, and how using a regular expression would make it a lot simpler.

Brushing aside the mental cobwebs I looked up a couple of references of the Java API and studied up on my friends Pattern and Matcher. I had trouble finding any examples for my specific case, where what I wanted to match spanned multiple lines. At first I thought I’d found the answer with the promising sounding Pattern.MULTILINE argument to Pattern.compile(), but that has to do with matching ^ and $. Without that option, those operators only match at the beginning or end of the text being parsed, with them it will allow them to work within the text at newline boundaries.

Turned out what I was looking for was the Pattern.DOTALL argument. By default, the dot operator does not match newlines, with this argument it does. An alternative is to prefix the regex pattern with (?s). It has the same effect, and the mnemonic stands for “single-line” mode, which is what its called in Perl.

So, to extract the relevant sections from the input above, you can do the following:

Pattern pattern = Pattern.compile(“—-START\n(.*?)\n—-END\n”, Pattern.DOTALL);
Matcher matcher = pattern.matcher(theInput);

while(matcher.find()) {
System.out.println(matcher.group(1) + “\n”);

That will print out:


anotherthing=ran outta foo words

You can get the same result with the alternate method, using the embedded (?s) operator in the Pattern declaration:

Pattern pattern = Pattern.compile(“(?s)—-START(.*?)—-END”);

For further reference, have a look here.