From a certain point of view, the amalgam of virtualization technology and information security techniques represents a rather strange blend. What does emulation or multiplexing of physical devices have to do with the enforcement of a wide variety of security properties?
The easiest answer (and the most traditional one) is that virtualization provides an effective means of isolating execution environments; virtualization seems like a natural way to provide isolation between execution containers. As we've seen previously, however, customizing the communication between such containers --- in many situations, they do need to communicate: complete isolation is the exception rather than the rule --- presents a challenge. Thus, even the "obvious" security application of virtualization is fraught with difficulty.
As organizations increase their adoption of virtualization environments, and with the current industry focus on information security, it is natural to wonder just how a virtualization framework might pull double duty by improving a security posture as well as easing management burden and infrastructure costs.
Besides isolation, virtualization frameworks seem to provide a natural place to implement a reference monitor: a formal, well-defined security construct. A reference monitor provides a low-complexity, trusted (and trustworthy) environment from which to observe the execution of another system and measure a certain set of security-related properties.
Unfortunately, it appears that little thought has been given to what the best way is to combine the twin roles of resource provider and reference monitor within a single virtualization framework. As a result, virtualization environments can find themselves attempting to measure security-relevant properties of a system in
ways that are both creative and convoluted. In essence, the set of events that are interesting from a security viewpoint (and this depends on what type of "security" you're interested in measuring...from integrity of control flow or data items to information flow to authorization and access control) are not necessarily the set of events that the virtualization framework was built to intercept and observe with a minimal performance impact.
Karger and Safford's article in the upcoming issue of IEEE S&P magazine details the I/O complexities of most of the popular approaches to providing virtualization. I and my colleagues Bratus, Ramaswamy, and Smith have a paper at the upcoming VMSec workshop (held with ACM CCS 2008) identifying the problem of designing an efficient event trapping system of use for both security policy enforcement and virtualization.
While the suggestion that the design of current virtualization solutions is actually a hindrance to providing security solutions may not sit well with folks interested in touting a particular virtualization solution's security capabilities, I would argue that we have a unique opportunity to make sure that VM platforms are designed to do the things we're asking them to do. Now is also a good time to note that the stunning complexity of VMM I/O subsystems, the performance hacks therein, and the backdoor management interface all suggest that even the basic isolation story rests on somewhat shaky ground.
We find ourselves at a unique point in time: we can try to identify the right design for doing these two disparate tasks at once, or we can muddle through by abusing a framework meant for resource multiplexing rather than program supervision. In either case, we still must balance the tradeoff between the virtualization framework's I/O architecture and subsystems and the trustworthiness of the reference monitor. Ironically, as we depend on VM frameworks to implement more security functionality, these systems become less trustworthy even as they become more trusted.
