I almost made it a year without a blog post.
I recently started using Vagrant to test our auto-provisioning of servers with Puppet. Having a simple-yet-configurable system for starting up and accessing headless virtual machines really makes this a much simpler solution than VMware Fusion. (Although I wish Vagrant had a way to take and rollback VM snapshots.)
Unfortunately, as soon as I tried to really do anything in the VM my
Mac would completely bog down. Eventually the entire UI would stop
updating. In Activity Monitor, the dreaded kernel_task was taking
100% of one CPU, and VBoxHeadless taking most of another. Things
would eventually free up whenever the task in the VM (usually
puppet apply) would crash with a segmentation fault.
Digging into this, I found an ominous message in the VirtualBox logs:
AIOMgr: Host limits number of active IO requests to 16. Expect a performance impact.
Yeah, no kidding. I tracked this message down to the "Use host I/O cache" setting being off on the SATA Controller in the box. (This is a per-VM setting, and I am using the stock Vagrant "lucid64" box, so the exact setting may be somewhere else for you. It's probably a good idea to turn this setting on for all storage controllers.)
When it comes to Vagrant VMs, this setting in the VirtualBox UI is not very helpful, though, because Vagrant brings up new VMs automatically and without any UI. To get this to work with the Vagrant workflow, you have to do the following hacky steps:
- Turn off any IO-heavy provisioning in your Vagrantfile
vagrant upa new VM
vagrant haltthe VM
- Open the VM in the VirtualBox UI and change the setting
- Re-enable the provisioning in your Vagrantfile
This is not going to work if you have to bring up new VMs often.
Fortunately this setting is easy to tweak in the base box. Open up
~/.vagrant.d/boxes/base/box.ovf and find the
You'll see an attribute
HostIOCache="false". Change that value to
Lastly, you'll have to update the SHA1 hash of the
.ovf file in
~/.vagrant.d/boxes/base/box.mf. Get the new hash by running
openssl dgst -sha1 ~/.vagrant.d/boxes/base/box.ovf and replace the
old value in
box.mf with it.
That's it. All subsequent VMs you create with
vagrant up will now
have the right setting.