An Exhaustive Survey on Security Concerns and Solutions at Different Components of Virtualization An Exhaustive Survey on Security Concerns and Solutions at Virtualization

National Institute of Technology Goa, India

ACM Comput. Surv., Vol. 52, No. 1, Article 12, Publication date: January 2019.

Virtualization is a key enabler of various modern computing technologies. However, it brings additional vulnerabilities that can be exploited to affect the availability, integrity, and confidentiality of the underlying resources and services. The dynamic and shared nature of the virtualization poses additional challenges to the traditional security solutions. This article explores the vulnerabilities, threats, and attacks relevant to virtualization. We analyze the existing security solutions and identify the research gaps that can help the research community to develop a secured virtualization platform for current and future computing technologies.

CCS Concepts: • Security and privacy → System security; Operating system security; Virtualization and security; Network security; Denial-of-service attacks;

Additional Key Words and Phrases: Virtualization, hypervisor, virtual machine, vulnerability, security

ACM Reference format:
Rajendra Patil and Chirag Modi. 2019. An Exhaustive Survey on Security Concerns and Solutions at Different Components of Virtualization. ACM Comput. Surv. 52, 1, Article 12 (January 2019), 38 pages.


Virtualization is an abstraction of hardware and software resources allowing heterogeneous architectures to run on the same hardware [9]. The major components of the virtualization are virtual machines (VMs), hypervisors, and virtual networks. VM is a software system similar to the physical system allowing us to run an operating system and other applications. Hypervisor offers an “abstraction of physical resources” like CPU, memory, network, and storage. It allows us to run multiple operating systems at a time on the same physical resources. It can be installed directly on hardware (Type I) or as a part of host OS (Type II). A virtual network enables communication among the VMs through virtual switch (vSwitch).

In past decade, virtualization-based applications have increased by a tremendous amount. However, the current implementations of the virtualization introduce many vulnerabilities and security challenges. In the National Vulnerability Database (NVD), several vulnerabilities related to well-known hypervisors such as XEN, VMware, and Qemu have been recorded. The successful exploitation of these vulnerabilities leads to many attacks, which can affect the confidentiality, integrity, or availability (CIA) of the hypervisor or its underlying VMs. For instance, critical VENOM vulnerability (CVE-2015-3456) in the open-source Qemu hypervisor allows an attacker to break out a VM, execute code on a host machine, and access all the other VMs on the host. This leads to hundreds or thousands of virtualization products being vulnerable to VENOM. The Foreshadow vulnerability (CVE-2018-3646) in XenServer allows an attacker to create a speculative side channel and steal data in VM RAM from other non-trusted VMs on the same physical server. Kaspersky Lab has analyzed that the cost of recovering from a security incident doubles when the attack affects virtual infrastructure. Thus, the virtualization environment needs to be secured.

This article explores various vulnerabilities, security threats, and attacks related to virtualization and classify them according to the core components of the virtualization infrastructure. We investigate the existing security approaches and techniques for each component and explore the research gaps. By considering these research gaps, we provide possible security research directives that will help the research community to address the needs of virtualization security and to design a secured and trusted virtualization platform for future computing technologies. Table 1 presents different key terms that we have used in this article.

Table 1. Key Terms
Term Description
Control VM It is a VM with higher privileges, e.g., Dom0 in Xen
Rouge VM VM with abusive administrator
Co-hosted VM VM which has shared resources with other VMs on the same physical server
Hypervisor Management Interface (HMI) It manages the hypervisor and VM building functionalities
Launching channel It is a communication channel from the origin of the VM image to the destination hypervisor where image is to be deployed
Hypervisor Layer This layer enables the VMs to run on a physical server
VM Management Layer It manages the Virtual Machines
VM (OS) Kernel Layer It builds the Virtual Machine
Virtual Machine Layer It contains all VMs on a single physical server
VM Network Layer It makes VMs to communicate with each other through virtual switch
Trusted Computing Base (TCB) It is a combination of hardware and software components that defines the trusted computing environment
Hardware VM (HVM) It require virtualization extensions from the host CPU and thus, using Qemu (Quick emulator), it emulates the physical hardware
Paravirtualized VM (PVM) It can run efficiently without hardware emulation.

In following, Section 2 discusses the vulnerabilities, threats, and attacks to virtualization. Section 3 analyzes the existing security solutions and challenges to the virtualization. Section 4 discusses the research gaps in the existing solutions and provides future research directives for offering better security in virtualization. Finally, Section 5 concludes our findings.


As shown in Figure 1, the virtualization environment includes the physical servers and other related components such as the management server, the VM repository, and networks. A physical server is virtualized using the hypervisor, which provides the abstractions of the physical resources. It has five components: VM, hypervisor, virtual network, host OS (control VM in the case of a Type 1 hypervisor), and underlying hardware. A hypervisor runs multiple VMs on the same hardware. A VM or applications running on VMs can be accessed by an end user through the Internet. A management server is used to handle the management-related tasks of the virtualization. The VM repository stores the VM images, which are managed through the interaction with the management server. A VM owner can launch a VM image to a hypervisor through the interaction with the management server. The network component presents typically three different networks: virtual network, internal network, and external network. VMs are connected over vSwitch, which forms the virtual network. The internal network is responsible for connecting the physical servers, storage servers, and management server. The external network connects the virtual environment to the external users through the Internet.

Fig. 1.
Fig. 1. Components of virtualization and attack surface.

Fig. 2.
Fig. 2. Taxonomy of virtualization-related attacks.

The security threats to the virtualization are due to the vulnerabilities existing in the different components and operations involved in the virtual environment. Figure 2 shows the taxonomy of virtualization-related attacks. An attack surface is mapped from Figure 1 to Figure 2 to discuss the possible attacks from a particular source to a victim.

2.1 Security Challenges to Hypervisor

Hypervisor enables the virtualization, and it has different components. For example, the Xen (Type 1) hypervisor contains a hypervisor core, management interface/privileged domain (Dom0), Qemu, and guest VMs. the KVM (Type 2) hypervisor interacts with the host OS, which is an additional component of KVM. The hypervisor core controls the execution of VMs by providing hardware abstraction. It performs CPU scheduling and memory partitioning for the VMs running on the hardware device. The hypervisor management interface carries all management actions of a VM. In a type 1 hypervisor, it is a control VM (Dom0), while in a type 2 hypervisor, it is a trusted process of the host OS. Dom0 is a privileged VM that has special rights to access I/O resources. Xen supports two types of VM: paravirtualized VM (PVM) and hardware VM (HVM). For guests running as HVMs, Xen uses Qemu, which runs in Dom0 to provide device emulation. For hypervisor versions after Xen 3.3, Qemu is considered as a large attack surface. Thus, it is moved outside of Dom0 and placed in stub domain for each guest. Unlike Xen, KVM relies entirely on Qemu for device emulation. It has been observed that an attacker can target each of these components. A compromised hypervisor can provide the control of all the underlying VMs. Thus, the most general way to attack a virtual environment is to gain access of the hypervisor.

2.1.1 Vulnerabilities. Hypervisor-related attacks are possible due to many vulnerabilities like uncontrolled flexibility to create VMs, misconfiguration, bugs and poor design, weak control over privileged and management interfaces, uncontrolled resource allocation to VM, and so on. In past 2 years (Jan 2016–Dec 2017), around 90 Xen hypervisor-related vulnerabilities have been recorded in NVD. We have classified Xen-related vulnerabilities by considering the source and target, as shown in Table 2. The root cause of Xen vulnerabilities is given in Table 3. In the past two years (Jan 2016–Dec 2017), around 174 Qemu+KVM–related vulnerabilities have been recorded in NVD. By considering guest VM and remote attacker as the sources and hypervisor, host OS, and guest VM applications as targets, we have classified these vulnerabilities, as shown in Table 4. The root cause of Qemu+KVM vulnerabilities is given in Table 5. Here, we have considered four types of vulnerabilities: Denial of Service (DoS), Gain Privilege (GP), Gain Information (GI), and Code Execution (CE).

Table 2. Source- and Targetwise Distribution of Xen Vulnerabilities
Source Target Type Vulnerabilities recorded in NVD
Guest User, Guest Kernel, Qemu Stub-domain Hyper-visor Dos CVE-2017-17566, CVE-2017-17565, CVE-2017-17564, CVE-2017-17563, CVE-2017-17045, CVE-2017-17044, CVE-2017-15597, CVE-2017-15596, CVE-2017-15595, CVE-2017-15594, CVE-2017-15593, CVE-2017-15592, CVE-2017-15591, CVE-2017-15590, CVE-2017-14431, CVE-2017-12136, CVE-2017-12135, CVE-2017-12134, CVE-2017-10923, CVE-2017-10922, CVE-2017-10921, CVE-2017-10920, CVE-2017-10919, CVE-2017-10917, CVE-2017-10914, CVE-2016-10025, CVE-2016-10024, CVE-2016-9818, CVE-2016-9817, CVE-2016-9816, CVE-2016-9815, CVE-2016-9385 CVE-2016-9383, CVE-2016-7154, CVE-2016-7094, CVE-2016-6259, CVE-2016-3960, CVE-2016-2270, CVE-2016-1571, CVE-2016-1570, CVE-2015-8615, CVE-2015-8552, CVE-2015-8551, CVE-2015-8550
GP CVE-2017-17566, CVE-2017-17564, CVE-2017-17563, CVE-2017-17045, CVE-2017-15595 CVE-2017-15594, CVE-2017-15592, CVE-2017-15590, CVE-2017-12137, CVE-2017-12136, CVE-2017-12135, CVE-2017-12134, CVE-2017-10914, CVE-2017-10913, CVE-2016-7093, CVE-2016-7092, CVE-2016-6258, CVE-2016-4962, CVE-2016-4480, CVE-2016-3960, CVE-2016-3157, CVE-2016-1570 CVE-2015-8554, CVE-2015-8550
GI CVE-2017-17046, CVE-2017-17045, CVE-2017-15597, CVE-2017-15589, CVE-2017-12855, CVE-2017-12135, CVE-2017-12134, CVE-2017-10917, CVE-2017-10916, CVE-2017-10914, CVE-2017-10913, CVE-2017-10911, CVE-2017-7995, CVE-2016-9932, CVE-2016-9384, CVE-2016-9383, CVE-2016-7154, CVE-2016-3159, CVE-2016-3157, CVE-2016-1570, CVE-2015-8553
CE CVE-2017-15588, CVE-2017-8905, CVE-2017-8904 CVE-2017-8903, CVE-2016-9383, CVE-2016-7154
Dom0, Tool-stack Dos/ GP/GI CVE-2016-9603, CVE-2017-2615, CVE-2017-2620, CVE-2016-5403, CVE-2016-9381, CVE-2016-9637, CVE-2015-8554, CVE-2015-7504, CVE-2017-14317, CVE-2016-4963, CVE-2016-4962, CVE-2016-5242
Guest VM Dos/ GP/GI CVE-2017-8905, CVE-2017-10911, CVE-2017-10916, CVE-2016-6259, CVE-2016-3158, CVE-2017-12134, CVE-2017-12855, CVE-2017-15589, CVE-2017-17046
Guest User Guest OS Dos/ GP/GI CVE-2016-2271, CVE-2016-3157, CVE-2016-3961, CVE-2016-4480, CVE-2016-3712, CVE-2016-7777, CVE-2016-9386, CVE-2016-9382, CVE-2016-9377, CVE-2016-9378 CVE-2016-10013

DoS: Denial of Service, GP: Gain Privileges, GI: Gain Information, CE: Code Execution

Table 3. Root Cause of Xen Vulnerabilities
Root cause Vulnerabilities recorded in NVD
Insufficient validation of guest input CVE-2015-8615, CVE-2016-4962, CVE-2014-3672, CVE-2016-6259, CVE-2016-9384, CVE-2016-9932, CVE-2016-9815, CVE-2016-9816, CVE-2016-9817, CVE-2016-9818
Insufficient bug check by hypervisor CVE-2016-1570, CVE-2016-4963, CVE-2016-5403, CVE-2016-7094, CVE-2016-9386, CVE-2016-9377, CVE-2016-9378, CVE-2016-10025, CVE-2017-10913, CVE-2017-10914, CVE-2017-10919, CVE-2017-14316, CVE-2017-14318, CVE-2017-14319, CVE-2017-15591, CVE-2017-17565
Incorrect error handling CVE-2017-10918 , CVE-2017-17044, CVE-2017-17045, CVE-2017-17563, CVE-2017-17564
Page fault CVE-2016-3961, CVE-2016-4480, CVE-2017-8903 , CVE-2017-8904, CVE-2017-10912, CVE-2017-10920,CVE-2017-10921,CVE-2017-10922, CVE-2017-15595, CVE-2017-15588, CVE-2017-15593, CVE-2017-17566
Shadow memory flaws CVE-2016-3960, CVE-2017-10915, CVE-2017-15592
PCI device config. error CVE-2015-8554, CVE-2017-15590
Bug in domain building CVE-2016-5242, CVE-2017-14317
Hypervisor memory corruption CVE-2016-3712, CVE-2016-7093, CVE-2016-7154, CVE-2016-9385, CVE-2016-9383, CVE-2016-9381, CVE-2016-10024, CVE-2016-9603, CVE-2017-8905, CVE-2017-12855, CVE-2017-15597
Unauthorized memory access CVE-2017-2615, CVE-2017-2620, CVE-2017-7228, CVE-2017-10911, CVE-2017-10918, CVE-2017-10923, CVE-2017-17046
Code emulation CVE-2015-7504, CVE-2016-7777, CVE-2016-9379,CVE-2016-9380, CVE-2016-10013
Data remanence CVE-2015-8555, CVE-2017-15589
Hardware fault CVE-2016-1571, CVE-2016-2271, CVE-2016-3158,CVE-2016-3159
Privilege escalation CVE-2016-3157, CVE-2016-6258, CVE-2016-7092, CVE-2016-9382, CVE-2016-9637, CVE-2017-12137, CVE-2017-15594
Other CVE-2017-10916, CVE-2017-10917, CVE-2017-12135, CVE-2017-12136, CVE-2017-12134, CVE-2017-15596
Table 4. Source- and Targetwise Distribution of Qemu+KVM Vulnerabilities
Source Target Type Vulnerabilities recorded in NVD
Guest User, Guest Kernel Hyper-visor Dos CVE-2017-17381, CVE-2017-15289, CVE-2017-13711, CVE-2017-13673, CVE-2017-13672, CVE-2017-12809, CVE-2017-11434, CVE-2017-10806, CVE-2017-10664, CVE-2017-9524, CVE-2017-9503, CVE-2017-9375, CVE-2017-7718, CVE-2017-6058, CVE-2017-5987, CVE-2017-5973, CVE-2017-5957, CVE-2017-5931, CVE-2017-5667, CVE-2016-10029, CVE-2016-10028, CVE-2016-9922, CVE-2015-8613, CVE-2016-9104, CVE-2016-8669, CVE-2016-8668, CVE-2016-8667, CVE-2016-8578, CVE-2016-8576, CVE-2016-7909, CVE-2016-7908, CVE-2016-7907, CVE-2016-7423, CVE-2016-7422, CVE-2016-7421, CVE-2016-7170, CVE-2016-7157, CVE-2016-7156, CVE-2016-7155, CVE-2016-6888, CVE-2016-6835, CVE-2016-6834, CVE-2016-6833, CVE-2016-6490, CVE-2016-6351, CVE-2016-5403, CVE-2016-5338, CVE-2016-5238, CVE-2016-5126, CVE-2016-5107, CVE-2016-5106, CVE-2016-4964, CVE-2016-4952, CVE-2016-4454, CVE-2016-4453, CVE-2016-4441, CVE-2016-4439, CVE-2016-4002, CVE-2016-4001, CVE-2016-3712, CVE-2016-2858, CVE-2016-2857, CVE-2016-2841, CVE-2016-2538, CVE-2016-2392, CVE-2016-2391, CVE-2016-1714, CVE-2016-1568
GP CVE-2017-8284, CVE-2016-9381
GI CVE-2017-15038, CVE-2016-9908, CVE-2016-9845, CVE-2016-9103, CVE-2016-6836, CVE-2016-5337, CVE-2016-5105, CVE-2016-4454, CVE-2016-4020
CE CVE-2017-14167, CVE-2017-7980, CVE-2017-5931, CVE-2017-5667, CVE-2015-7504, CVE-2014-0145, CVE-2016-7161, CVE-2016-6351, CVE-2016-5338, CVE-2016-5126, CVE-2016-4439, CVE-2016-4002, CVE-2016-3710, CVE-2016-1714, CVE-2016-1568
Host Dos/ GP/ GI CVE-2017-9374, CVE-2017-9373, CVE-2017-9330, CVE-2017-9310, CVE-2017-9060, CVE-2017-8379, CVE-2017-8309, CVE-2017-8112, CVE-2017-8086, CVE-2017-7980, CVE-2017-7377, CVE-2017-6505, CVE-2017-5857, CVE-2017-5856, CVE-2017-5579, CVE-2017-5578, CVE-2017-5552, CVE-2017-5526, CVE-2017-5525, CVE-2016-10155, CVE-2015-8568, CVE-2015-8567, CVE-2016-9916, CVE-2016-9915, CVE-2016-9914, CVE-2016-9913, CVE-2016-9106, CVE-2016-9105, CVE-2016-9102, CVE-2016-9101, CVE-2016-8910, CVE-2016-8909, CVE-2016-8577, CVE-2016-7995, CVE-2016-7994, CVE-2016-7466, CVE-2016-5403, CVE-2016-4037, CVE-2015-8558, CVE-2015-1779
Guest User Guest VM Dos/ GP/ GI CVE-2017-11334, CVE-2017-5898, CVE-2015-8619, CVE-2015-8504, CVE-2015-8345, CVE-2015-7504, CVE-2014-0146, CVE-2014-0145, CVE-2014-0143, CVE-2014-0142, CVE-2015-8818, CVE-2015-7512, CVE-2015-5158

DoS: Denial of Service, GP: Gain Privileges, GI: Gain Information, CE: Code Execution

Table 5. Root Cause of Qemu+KVM Vulnerabilities
Root cause Vulnerabilities recorded in NVD
Out-of-bounds read access CVE-2017-13672, CVE-2017-11434, CVE-2017-7718, CVE-2016-10029, CVE-2016-10028, CVE-2016-9104, CVE-2016-8668, CVE-2016-5107, CVE-2016-4454, CVE-2016-3712
Out-of-bounds array CVE-2017-16845, CVE-2017-11334, CVE-2017-6058, CVE-2016-4952
Divide-by-zero error CVE-2017-17381,CVE-2016-9922,CVE-2016-8669,CVE-2016-8667
Out-of-bounds write access CVE-2017-15289, CVE-2015-8619, CVE-2016-7423, CVE-2016-7170, CVE-2016-6351, CVE-2016-5238, CVE-2016-5106, CVE-2016-4441, CVE-2016-4439, CVE-2016-1714
Remote VNC client CVE-2017-15124 , CVE-2017-7980
Illegal access to host heap memory CVE-2017-15038, CVE-2017-5931, CVE-2017-5667, CVE-2016-9103, CVE-2016-7161, CVE-2016-4020, CVE-2016-2857
NULL pointer dereference CVE-2017-12809, CVE-2017-9503, CVE-2014-0146, CVE-2016-8578, CVE-2016-7422,CVE-2016-2392, CVE-2016-2391, CVE-2016-1922
Infinite loop CVE-2017-9330, CVE-2017-9310, CVE-2017-6505, CVE-2017-5987, CVE-2017-5973, CVE-2015-8345, CVE-2016-9776, CVE-2016-8576, CVE-2016-7909, CVE-2016-7908, CVE-2016-7421, CVE-2016-7156, CVE-2016-7155, CVE-2016-6834, CVE-2016-6490, CVE-2016-4964, CVE-2016-4453, CVE-2016-4037, CVE-2016-2841, CVE-2015-8558
Memory consumption CVE-2017-9374, CVE-2017-9373, CVE-2017-9060, CVE-2017-8379, CVE-2017-8309, CVE-2017-8086, CVE-2017-7377, CVE-2017-5857, CVE-2017-5856, CVE-2017-5579, CVE-2017-5578, CVE-2017-5552, CVE-2017-5526, CVE-2017-5525, CVE-2016-10155, CVE-2015-8568, CVE-2015-8567, CVE-2016-9916, CVE-2016-9915, CVE-2016-9914, CVE-2016-9913, CVE-2016-9106, CVE-2016-9105, CVE-2016-9102, CVE-2016-9101, CVE-2016-8577, CVE-2016-7995, CVE-2016-7994, CVE-2016-7466, CVE-2016-5403
CPU consumption CVE-2017-8112, CVE-2016-8910, CVE-2016-8909, CVE-2016-4964, CVE-2016-4037
Qemu Device memory leakage CVE-2016-9912, CVE-2016-9912, CVE-2016-9908, CVE-2016-9907,CVE-2016-9846,CVE-2016-9845,CVE-2016-2538,CVE-2015-8743,CVE-2015-8701
Qemu emulator DoS CVE-2016-2198, CVE-2016-2197, CVE-2016-1981,CVE-2016-1568, CVE-2015-8745

2.1.2 Threats. The vulnerabilities in a hypervisor can be exploited through external interfaces of the hypervisor resulting into some common threats: insertion of malware or rootkits, management interface compromise, uncontrolled growth of VMs, unauthorized access to hypervisor resources, denial of service through resource starvation by VM, network threats, launching rouge VM, and so on. A threat model to hypervisor is shown in Figure 3. A hypervisor interacts with the following different components: hypervisor code, hypervisor management interface/control VM/insider, VMs, networks, and VM user. A hypervisor code may be infected with the rootkits, which can violate the integrity of the hypervisor. A control VM may be exposed to different interfaces such as drivers, insider, and remote access. A VM, malicious insider, or a remote attacker can compromise the control VM. It can lead to taking control over management functionalities of the hypervisor. A hypervisor runs the VMs, which may be exposed to the Internet. A running VM can violate the confidentiality, integrity, as well as availability of the hypervisor. Since hypervisor implements a VM migration functionality, a malicious hypervisor in the network may compromise its migration process. A VM user may be malicious and can launch rouge VM on the hypervisor.

Fig. 3.
Fig. 3. Threat model for hypervisor.

2.1.3 Attacks. Since the hypervisor is the core component in virtualization, the attacks related to it have a serious impact on virtualization security.

  1. Hyperjacking through VM-based rootkit (VMBR). This aims to violate the integrity of the hypervisor and is typically launched against type 2 hypervisors. Here an attacker attempts to get control over a hypervisor. Through hyperjacking, an attacker can inject a VMBR between hardware and the OS. It can run as a thin rogue hypervisor under the legitimate infrastructure (hypervisor or OS). Such a rogue hypervisor can take complete control of the host OS or hypervisor to control all the interactions between legitimate hypervisor and hardware. For example, SubVirt [55] installs a hypervisor below the host OS and controls the VM. Blue Pill [32] exploits hardware extensions in the virtualization enabled CPUs and runs an infected system into a VM.
  2. Attacks from the compromised management interface/Malicious insider. In Xen HVM mode and in KVM, Qemu flaws affect Xen security. A privileged user with the guest VM can target vulnerabilities (e.g., heap overflow) in the device model process (Qemu) running in Dom0 and can gain privileges of Dom0 (CVE-2016-9603) and (CVE-2017-2615). Thus, the recent research [30, 97] states that the Dom0 cannot be considered in the hypervisor TCB. Thus, different attacks are possible to the hypervisor from the Dom0.
    A. Compromising CIA of DomUs. A VM user always trust the VM provider and thus the hypervisor. If Dom0 is compromised, then it can compromise the CIA of the DomUs.
    B. Direct Memory Access (DMA) attack. This aims to violate the integrity of hypervisor. Since Dom0 can communicate with hardware devices directly, it can access the main memory of hypervisor through DMA transfer. If the physical memory address of the hypervisor is known to Dom0, then it can read and write the data structures of the hypervisor. In addition, it can overwrite parts of the main memory to insert the rootkit.
    C. VM sprawl attack. This aims to violate the availability of the hypervisor. A hypervisor offers the flexibility to create new VMs. However, such flexibility carries a potential risk, like VM sprawl. A malicious insider with an administrative access to management interface can create large numbers of VMs. Thus, at a particular time, large numbers of VMs are very difficult to monitor and manage.
  3. Attacks from the VM. Here we assume that a VM image launched to the hypervisor is secure. However, the risk to the hypervisor remains due to the insecure interface or malicious user at the running VM. Table 2 and Table 4 show most of the hypervisor vulnerabilities that are exploited from the guest VM.
    D. Code execution on Hypervisor. A malicious VM user can modify arbitrary memory (heap memory) of hypervisor and perform arbitrary code execution.
    E. VM (to Hypervisor) escape. It takes an advantage of isolation failure between the hypervisor and VMs. Here a rouge VM completely bypasses the access control function of the hypervisor and gain privileges of the hypervisor. Thus, a rouge VM can control other VMs and the hypervisor itself.
    F. Hypervisor crash (DoS). A VM may corrupt the hypervisor memory and cause the hypervisor to crash.
    G. Hypervisor information leakage. A VM can arbitrarily access the hypervisor memory, which results in information leakage of the hypervisor and other VMs.
    H. Resource starvation. This aims to violate an availability of the hypervisor. The uncontrolled resource allocation allows a VM to consume the resources of the physical server, which results into a denial of service to other VMs. It makes physical resources unavailable to other VMs by flooding a large number of requests.
    I. Attack on control VM. As control VM shares the resources with DomUs, it can be attacked from the DomU. It aims to take control of the hypervisor.
    J. Attack on other VMs (VM hopping). It is a process of jumping from one VM to another VM on the same physical server by exploiting hypervisor vulnerabilities. The arbitrary access to hypervisor resources (e.g., memory) may result into attacks like information leakage, traffic analysis, modification of the configuration file, denial of service, and so on, on other VMs. The authors of Reference [49] have demonstrated the VM hopping by maliciously gaining an access to different VMs.
  4. Attacks from the malicious hypervisors (networks). The migration module in hypervisor implements the VM migration functionalities. The vulnerabilities like buffer overflow, stack overflow, and so on, in the migration module or bugs in the authentication and access control mechanisms can be exploited through the network to compromise the control plane of a migration process and to perform following attacks.
    Migrating a compromised VM. For dynamic load balancing, a live migration is automatically initiated. An attacker at malicious hypervisor may falsely advertise available resources and influence the control plane to migrate the compromised VM to the trusted hypervisor.
  5. Launching rouge VM. A malicious VM user may launch a malware-infected VM image on a trusted hypervisor. It introduces the new source of an attack in the virtual environment when underlying infrastructure is secured.

2.2 Security Challenges to VM

A VM is a potential target to gain an unauthorized access to the hypervisor and other VMs. At particular time, a VM can be in one of the three states: inactive, moving (migrating to other host machine), and running state. In all these states, it is vulnerable to different types of attacks. NVD has recorded following vulnerabilities related to VM: “Xen 3.2.x through 4.4.x does not properly clean memory pages that are recovered from guests, which allows local guest OS users to obtain sensitive information via unspecified vectors (CVE-2014-4021).” “Apache CloudStack before 4.5.2 does not properly preserve VNC passwords when migrating KVM virtual machines, which allows remote attackers to gain access by connecting to the VNC server (CVE-2015-3252).” “BIG-IP and BIG-IQ AWS, Azure, or Verizon cloud instances do not properly regenerate certificates and keys when deploying VM image, which makes multiple instances to share the same certificates and keys. It causes the disruption of services and/or an information leak (CVE-2016-2084).”

2.2.1 Security Challenges to VM in Running State. In virtualization, a VM in the running state is a potential target since it has interfaces that are exposed to an attacker.

Vulnerabilities. VM in running state includes vulnerabilities like poor isolation between VM and hypervisor, poor access control over management interface, uncontrolled VM rollback, default state of new VMs, poor isolation for shared resources, network vulnerabilities, and so on.

Threats. The possible threats are rootkit insertion in a VM, illegal access from the hypervisor management interface or a malicious insider, network threats, threats from the rouge VM, isolation failure among the VMs. A threat model to the VM running state is shown in Figure 4. A running VM interacts with different components: Hypervisor, Hypervisor management interface, VM kernel code, Co-hosted VMs and Networks. A hypervisor may be exposed to perform different attacks on the VMs. A control VM may be exposed to different interfaces such as drivers, insider and a remote access interface. A VM, malicious insider or a remote attacker can compromise the control VM. A VM instance may be created from the rootkit infected VM image that can tamper the kernel of a VM. A VM kernel can be compromised,which can tamper with the execution of a trusted application. A VM co-hosted with other VMs may be exposed to the Internet. A co-hosted VM can perform different attacks on other VMs that share the underlying resources. A VM network traffic may be exposed to a network attacker.

Attacks on a VM in running state.

Fig. 4.
Fig. 4. Threat model for VM running state.
  1. Attacks from the compromised hypervisor. Here we consider both type 1 and type 2 hypervisor. In case of type 1 hypervisor, we assume that the control VM is in the TCB of the hypervisor. The VMs running on a compromised hypervisor are under the security risk. Here CIA of the VMs can be compromised.
  2. Attacks from the compromised management interface/malicious insider. The trusted computer base (TCB) of type 1 hypervisor includes not only the hypervisor but also the control VM. Control VM can control the access to the physical resources by other VMs (DomUs) and can control all these DomUs. Thus, we assume that the hypervisor is trusted; however, the code instantiating the functionality of the Dom0 contains bugs. In addition, in case of both the hypervisors, the remote access of management interface can be compromised through the web application attacks (CSS and SQL injection). It exposes the VM management actions to a remote attacker. For example, “CSS vulnerability (CVE-2012-5050) in VMware vCenter Operations before 5.0.x allows remote attackers to inject arbitrary web script via unspecified vectors and take control of vCenter.” In addition, management interface can be directly accessed by a malicious insider [52, 84]. The successful compromise of a management interface can lead to following attacks on the VMs [56].
    K. VM privacy violation. Since Dom0 has read access to the memory of a running VM, it is possible to extract sensitive information in clear text through memory snapshots. A malicious Dom0 allows us to get passwords, cryptography keys, files, and other confidential data [84]. It uses a cold boot attack on the memory snapshot of the target VM. The permissive access by Dom0 to the VM memory allows a malicious insider to capture the information in regular intervals to read the data exchanged between VMs [85].
    L. VM integrity violation. Since Dom0 has write access to the memory of a running VM, it can inject arbitrary code in a VM.
    M. VM network traffic sniffing/spoofing. Dom0 can intercept all network traffic before it gets to DomU. Here network traffic of a particular VM can be sniffed to read the communication with the paired destination or to perform MITM attacks.
    N. VM rollback. The rollback operation allows us to restore a VM to the previous state. It can disable the security measures that were previously enabled. In a VM rollback attack, a compromised hypervisor management interface executes a VM from the previous snapshot [129]. An attacker can use a brute-force technique to find the login password of a VM.
    O. VM relocation. With the evolution of integrity protected hypervisors, the features like memory snapshots and disk accesses are disabled. However, for the essence of load balancing, virtualization provides a functionality where VM can be relocated to another server. The attacker can find a malicious server to relocate the VM, where it is possible to take memory snapshot of a VM [95].
    P. VM cloning. As VM is a software, it can be backed up, copied, and moved to other servers. A VM owner is unaware of the number of running instances of a VM. A malicious hypervisor management interface can copy the VM and start it on a compromised hypervisor. It can have complete access to the VM.
  3. Kernel-level attack. A VM instance created through an infected VM image allows viruses, rootkits, and other malware to do substantial damage to a VM.
    Q. Kernel rootkits. It may get an administrative privileges and can tamper the VM kernel. Here, we see the severity of the kernel rootkits.
    Kernel code modification. An attacker can modify the OS kernel to insert rootkits. The goal of such modification is to run the rootkits at an administrator level.
    Kernel data structure modification. A rootkit can use memory corruption vulnerability to corrupt the data. It modifies the pointers and data directly in the kernel data structures. Direct Kernel Object Modification (DKOM) attack [16] changes the accounting table information in the kernel and completely bypasses the object manager. Direct Kernel Structure Manipulation (DKSM) attack [8] exposes the security code hosted in a VM by changing the syntax of the kernel structure.
    Kernel control-flow modification. A return-oriented rootkit [22] can bypass the code integrity protection and compromises the kernel code execution (control flow) of the running processes. Control-flow hijacking attack exploits a memory corruption bug to execute a malicious code in address space of a valid user process. An attack like binary code reuse redirects the control-flow to the existing executable code.
    R. Iago attack. In rootkit-based attack, it is assumed that the kernel is trusted, and rootkits may tamper its execution. However, commodity OS kernels are frequently prone to compromise and, thus, the reverse is also true, where kernel is untrusted and may tamper the execution of a trusted application. In an Iago attack [23], a malicious kernel makes an application to act against its interests by communicating with it. Applications interact with the kernel through the system call interface. Legacy applications generally do not check return values from the kernel. Thus, kernel can return crafted values (e.g., new memory mappings) to corrupt application behavior.
    S. Hypervisor to application tampering. The applications can be tampered by an attacker having an access to the hypervisor.
  4. Illegal access from the co-hosted VMs. A VM isolation failure can lead to the unexpected side channel and covert channel among the VMs. The vulnerabilities exposed due to the shared resources allow an attacker to co-locate a VM with the target the VMs and then to build a side/covert channel with target VMs. The co-hosted VMs are also vulnerable to the RowHammer attack [94]. The repeated access to a memory can cause bit flips in adjacent rows that may belong to another VM.
    T. VM side-channel attack. A side channel allows us to extract confidential information of the target VM. It uses the shared resources such as the CPU load, cache, memory, and so on, to extract the resource consumption status, cryptography keys, and other sensitive information of a target VM. Here an attacker does not need to share the address space with the target VM; rather, he or she shares hardware caches to access its memory.
    To enable side-channel attacks, it is required to detect the co-location of the target VM. The authors of Reference [83] have identified whether a particular VM is likely to reside on then same physical server. In addition to co-location detection, L2 cache-based side-channel attack is demonstrated, which causes information leakage across VMs on Amazon EC2. Similarly, HomeAlone [139] have used Prime+Probe technique on L2 cache to detect co-location on Xen. The authors of Reference [140] have extracted the ElGamal secret key that is used for GNU Privacy Guard decryption performed in another VM by monitoring the L1 cache timing. The authors of Reference [121] have targeted a cache timing attack to extract AES keys of a VM running on an ARM Cortex-A8 processor. The authors of Reference [39] have demonstrated a side-channel attack to recover AES keys in Xen and VMWare.
    L1 and L2 cache-based attacks are extremely difficult to perform, as these caches are private to the cores and thus, an attacker needs to exploit the vulnerability of the hypervisor scheduler to perform cross-VM attacks. In addition, attacker VM and target VM should co-reside on the same core. However, as the “last-level cache (LLC)” is shared among all cores, LLC-based side-channel attacks can be performed when the victim and attacker are not executing on the same core. Flush+Reload [134] observes the information leak through shared pages in Intel X86 processors and extracts GNU Privacy Guard private keys across multiple processor cores and across VMs. The authors of Reference [11] have used a combination of Flush+Reload and lattice techniques. Flush+Reload technique obtains the information and the lattice techniques to reconstruct secret keys for 256 bit elliptic curves. The Flush+Reload technique is employed in the VMware to recover the AES keys of “OpenSSL 1.0.1” running inside a target VM [47]. It takes an advantage of memory deduplication in the VMware. The authors [46] have exploited access time variations on the L3 cache. Here, huge pages are exploited to work across VM boundaries without requiring the memory deduplication. It requires only machine co-location in a multi-core scenario. It is a variation of L1 cache-based Prime+Probe attack and works like L3 cache-based Flush+Reload attack. The authors of Reference [45] have monitored L3 cache to achieve co-location. Here Prime+Probe technique is used to extract noisy data from a “Amazon EC2” VM. It processes the noisy data to obtain 2048-bit RSA encryption key. Prime+Probe attack translates the virtual addresses to physical addresses. It checks an access to these addresses through another VM to leak the VM information [135]. A Prime+Probe attack against the L3 cache on GnuPG can be used to extract the key of GnuPG crypto software [62]. A Flush+Flush attack [41] uses the execution time of the flush instruction. Since it does not make any memory accesses, there are no cache misses. The constant cache flush reduces the number of cache hits. Therefore, this attack cannot be detected using cache hits and misses. A shared memory-based side-channel attack exploits the vulnerabilities of the balloon driver, which is used by the hypervisors to balance physical memory among the VMs [119].
    U. VM covert-channel attack. It transfers confidential information, while violating the specified security policies. The authors of Reference [71] have presented a CPU load-based covert-channel attack. The authors of Reference [133] have used L2 cache to build a covert channel to steal the private keys. The authors of Reference [124] have introduced a shared memory-based covert channel in the Xen. The authors of Reference [130] have presented the memory deduplication-based covert-channel attack. Such covert channels are faster than the L2 cache-based covert channels. The authors of Reference [127] have presented a memory bus-based high bandwidth covert channel on the virtualized x86 platform. The authors of Reference [64] have introduced a LLC-based covert channel using Prime+Probe technique.
    V. RowHammer attack. The authors of Reference [94] have exploited the RowHammer bug in the DRAM devices to gain kernel privileges of same VM. The Rowhammer attack can be performed across the VMs by exploiting the memory de-duplication and flipping memory bits in a controlled way [79]. A malicious VM exploits bit flips induced by RowHammer attack to crack memory isolation enforced by virtualization [131]. It exploits the Xen paravirtualized memory isolation to access arbitrary physical memory of the target VM. In addition, it can exploit bit flips in random memory locations. It can create bit flips in cryptographic keys that are stored in memory [12]. For this, it requires only user-level privileges for kernel and memory location of the key. It combines timing Prime + Probe attack and row-buffer collision to induce bit flip faults in a 1,024-bit RSA key on modern processors. The authors of Reference [75] have performed side-channel and covert-channel attacks across processors using RowHammer. The authors of Reference [27] have performed a memory ambush attack similar to a RowHammer attack, which places the hammerable kernel buffers physically adjacent to the target objects (VM memory).
  5. Classical network attacks. The newly joined VM typically starts with the open ports, default services, and active protocols, which allows an attacker to perform port scanning, ARP spoofing, IP spoofing, network sniffing, DDoS, and so on.

2.2.2 Security Challenges to VM in Moving (or Migrating) State. A live migration of VM is an important feature for load balancing, hardware maintenance, and so on. However, it is susceptible to severe security risks such as a network sniffing, malicious code injection. An attacker may place himself in the migration transit path and can perform a MITM attack.

  1. MITM attack. The migration traffic can be evesdropped, which results into the leakage of current status of the running applications and sensitive information. In addition, it is possible to modify VM contents and to inject rootkits in VM code.

2.2.3 Security Challenges to a VM in Inactive State. A VM image in the repository, during launching or its data on a physical device, is considered as an inactive VM. It includes configuration files to create the VM instances.

Vulnerabilities. A VM image in the virtual environment has many vulnerabilities like weak access control, insecure launching channel, untrusted hypervisor, and so on. The presence of data on a physical device after VM release is considered as an additional vulnerability.

Threats. The most common threats to VM inactive state are the uncontrolled upload, creation, modification, and usage of VM images. Some others include unauthorized access to a launching channel and a physical device, deployment of the image to an untrusted hypervisor, and so on. A malicious modification to VM image and storing images with malicious code are serious threats. A threat model to VM inactive state is shown in Figure 5. VM inactive state can be considered through four different states: image in the repository, launching image, running image, and image removal in the virtual environment. The VM images in the repository can be exposed to the external users due to the weak access control and authentication. The VM image launching channel may be exposed to the network, where an attacker can perform a MITM attack. A VM image may be launched to an untrusted hypervisor. Once VM image is removed from the virtualization platform, its hardware can be given to another VM. Consider a new user of VM is malicious, who can violate the confidentiality of the removed VM data. In addition, the outdated software and known vulnerabilities of image can be exploited. As per VM image state, different attacks are possible.

Fig. 5.
Fig. 5. Threat model for VM inactive state.
  1. Attacks on VM image contents
    Exploiting outdated software and OS vulnerabilities. Once a VM is created from the image, the outdated software and known OS vulnerabilities can be exploited by an attacker through other VMs or virtual networks. For example, “Directory traversal vulnerability in the TS WebProxy in latest Microsoft OSes like Windows 7 SP1, Windows 8, Windows Server 2012 Gold and R2, etc., allows remote attackers to gain privileges (CVE-2015-0016).”
  2. Attacks on a VM image in repository
    Leakage of sensitive information. A user with a valid account at repository can access all images in the repository. He may create a VM instance from the VM image and may recover the passwords and cryptography keys of the previous user, which results into information leakage [42, 120].
    Modification of the VM image. The legitimate VM image can be infected through the insertion of rootkits or malware [40, 74]. A VM instantiated through such an image can introduce malware in virtual environment. It can be leveraged to analyze the VM activities and data, which results in a VM privacy breach.
    Malware infected image upload. An attacker can upload a VM image with malware to a public repository due to the lack of proper access control [40, 74, 120].
    VM image sprawl. As creating a new VM image is just a code, multiple images can be created in the repository through cloning, version, archives, and so on. Here an attacker can create such multiple copies to make the maintenance and security very difficult. This problem is known as VM image sprawl [80].
  3. MITM attack on VM image. A MITM attack is possible on a VM image during its launching from repository to destination hypervisor [74].
  4. Attack on VM image at destination hypervisor. A VM image of a user may be launched on an untrusted hypervisor. The malicious hypervisor may falsely advertise the available resources and make the VM owner deploy an image on it [74]. Since the VM owner cannot determine the trustworthiness of the destination hypervisor, his/her VM instance may run at an unsecured platform.
  5. VM data remanence attack. Data remanence is the physical representation of digital data that remains on the physical device even after removal of it. If the same hardware is allocated to an attacker, then VM data remanence allows an attacker to reconstruct the sensitive information of the previous VM. It is possible that data may present on both VM memory and the hard disk [2].

2.3 Security Challenges at Virtual Network

Network virtualization allows us to create and run multiple independent virtual networks over the shared infrastructure. In certain situations, the virtual network needs to be reconfigured dynamically. For example, when a VM migrates among the physical servers, its IP needs to be changed and corresponding VLANs need to be reconfigured. This makes virtual networks very complex. A software-defined network (SDN) makes such network reconfigurations easy where it separates the control functions of the network switch and places it in the centralized control plane [48]. It helps to control the data plane of network switches and routers in a software-defined manner. However, the sharing of network infrastructure increases the vulnerabilities in DNS servers, DHCP, IP, ARP protocols, and so on. In addition, there are many vulnerabilities such as vSwitch software bugs, open ports, insecure network channels, and so on.

  1. Network attacks. The exploitation of network vulnerabilities leads to different traditional attacks such as Denial of Service (DoS), port scanning, sniffing, IP/MAC spoofing, and so on.


There have been many works to date for virtualization security. We classify the existing solutions according to the security of each component in virtualization.

3.1 Hypervisor Security

Hypervisor has an interface to its code, management VM, hosted VMs, networks, and VM user (least trusted interface for VM provider). As shown in the threat model of hypervisor (see Figure 3), hypervisor can be attacked from all these interfaces. While protecting hypervisor from particular interface, it is assumed that inner interfaces are trusted and can be used to develop a security solution. Hypervisor code is protected using hardware or a nested hypervisor.

3.1.1 Securing Hypervisor from Rootkits through Integrity Checking. HyperGuard [86] uses SMM of the x86 processor to inspect the integrity of the hypervisor. SMM protects the integrity measurement code using hardware. Similarly, HyperCheck [111] prevents rootkits that target Xen hypervisor integrity. It outsources the analysis of integrity state. In addition, it can monitor the code and data of the control VM and other guest VMs. In HyperGuard [86] and HyperCheck [111], the measurement agent in the SMM can be tampered.

To protect tampering of the measurement agent, HyperSentry [6] uses a software-based integrity measurement agent with an access to the contextual information (correct CPU state) of hypervisor. The agent runs in the hypervisor context and isolated from the hypervisor by adopting a TCB that is composed of hardware and a software component. However, it finds the traces of an attack that has been happened.

HyperGuard [86], HyperCheck [111], and HyperSentry [6] are SMM dependent, which can be attacked by the SMM rootkits. In addition, as such a solution provides static hypervisor code integrity, it can be bypassed by the return-oriented rootkits. HyperSafe [116] verifies the control-flow integrity of type 1 hypervisor execution. It includes two techniques: Non-bypassable memory lockdown and Restricted pointer indexing. Non-bypassable memory lockdown provides hypervisor code integrity by locking down memory pages of hypervisor code and static data. The restricted pointer indexing converts the control data into pointer indexes that strictly follow the hypervisor control flow graph. It protects the control data for control-flow integrity by extending the code integrity. The authors of Reference [145] have measured a checksum of the runtime hypervisor by performing CPU microcode modification. MGUARD [63] prevents any illegal modifications of the hypervisor code. The advanced memory buffer of “FB-DIMM” can be used to integrate a programmable guard. The programmable guard continuously monitors the memory traffic between CPU and physical memory.

HyperGuard [86], HyperCheck [111], HyperSentry [6], HyperSafe [116], and MGUARD [63] focus on hypervisor control data and cannot protect hypervisor non-control data. HyperVerify [33], a VM-assisted architecture, monitors hypervisor non-control data using a control VM. The hardware state of hypervisor is captured in control VM using programmable device driver. It translates the low-level hardware state into the high-level hypervisor context using a memory analysis library. As per our observation, hypervisor should have an integrity checking mechanism during its deployment and runtime.

3.1.2 Securing Hypervisor from Compromised Management Interface/Malicious Insider. Hypervisor needs to be protected from the management interface, which can be either compromised by a VM or accessed by a malicious insider/remote attacker. In first case, the research reveals a new tread of disaggregating the management interface of hypervisor. It aims to provide a separate management interface to each VM; however, the execution environment may remain the same. In the second case, researchers use the logging and auditing mechanisms.

Disaggregating the management interface. The authors of Reference [68] have moved the VM-building functionality into a separate privileged VM that runs along with Dom0. Nova [103], a thin hypervisor, moves the most hypervisor functionality to the user level. It uses the principle of least privilege to reduce a TCB. Xoar [30] modifies the original Xen design by breaking the control VM into multiple unprivileged service VMs. It modifies the Xen hypervisor without compromising its functionality. It breaks the control VM into multiple service VMs with least privilege. Self-service computing (SSC) [17] splits Dom0 into a systemwide Dom0 and a VM-specific Dom0. It allows access to the VM-specific Dom0 and, thus, hypervisor functionality without breaking the isolation between VMs. HyprBIOS [141] shifts the control VM from the hypervisor TCB. The reduced TCB is further decomposed into two layers. The slave layer is a original hypervisor and runs the VMs; however, it passes the crucial VMEXIT handling to the master layer at SMM. The master layer checks the access control violation by VMEXIT. Privilege separation virtual machine (PSVM) security model [136] allocates a separate Dom0 to each VM. The authors of References [17, 30, 68, 103, 136, 141] have secured hypervisor functionalities, leaving the hypervisor still vulnerable through network drivers, and so on. Nexen [97] decomposes Xen into multiple domains such as “a privileged security monitor,” “a shared service domain,” and multiple VM specific Xen slices. Each VM slice is made up of a duplicated Xen code and data. A VM with compromised slice cannot directly access data of other VMs, and, thus, it cannot affect rest of the VMs or hypervisor.

Logging and auditing. Although TCB reduction approaches reduce the risk of accessing the hypervisor functionalities by a VM, they cannot protect them from the malicious insider. TVGuarder [58], a trace-enabled protection, detects malicious access to hypervisor functionalities by an insider. It observes the traces of VM operations and allows only legitimate VM operations. It can protect the VM sprawl attack on the hypervisor.

3.1.3 Securing Hypervisor from Rouge VMs. Here, a primary objective is to restrict any illegal access to hardware or hypervisor code from a VM. It aims at either provisioning strong isolation to hypervisor resources from VMs or an isolated execution environment for a VM. sHype [89] adds hooks in the hypervisor to enforce information flow constraints between VMs. It aims to reduce the VM interface to shared resources through the controlled resource sharing between VMs. NoHype [104] provides a multi-tenant architecture to ensure isolation between VMs. It aims to build an isolated execution of VM by providing fixed cores, pre-allocated memory to a VM. Similarly to NoHype, TrustOSV [113] builds an isolated execution environment for VM using hardware. Unlike NoHype, it minimizes the interaction of management OS with the VMs. Min-V [70] uses a delusional boot mechanism to restrict the boot time initialization code or earlier OS code. It aims to reduce the interfaces between the hypervisor TCB and VMs. It disables non-critical virtual devices offered to the VMs.

In case of type 2 hypervisor, it is focused on providing an isolated hypervisor environment for each VM. Thus, even if hypervisor is compromised through a VM, it will not affect other VMs. SplitVisor [73] has two layers: SplitVisor and GuestVisor. SplitVisor resides in a trusted computing base and provides isolation between VMs. GuestVisor offers virtual resources to the VMs. HyperLock [118] provides hypervisor isolation using separate “address space and restricted instruction set.” In addition, it creates a separate shadow hypervisor for each VM so that compromised hypervisor can affect only the corresponding VM. DeHype [123] reduces the exposed attack surface of a hosted hypervisor by deprivileging it to user mode. It decouples the hypervisor code from the host OS and makes the hypervisor as a user-level library. It allows concurrent execution of multiple hypervisors on the same physical machine. Similarly, in the case of Xen, Nexen [97] decomposes the Xen to provide a separate execution environment for each VM.

3.1.4 Securing Hypervisor from Networks (Other Hypervisors). Here it is necessary to secure a hypervisor when it communicates with other hypervisors for VM migration. The authors of Reference [112] have verified the destination hypervisor before it initiates the VM migration. A remote attestation protocol [18] attests the destination hypervisor where VM is to be migrated. The authors of Reference [109] have performed a property-based attestation of destination hypervisor. The authors of Reference [3] have attempted to guarantee the VM migration to a trustworthy hypervisor.

3.1.5 Securing Hypervisor from Malicious VM User. It aims to secure hypervisor from an untrusted launch of a VM image. Cloud Verifier (CV) [92] protects the integrity of VM and host. Here, the integrity of CV is verified by cloud users, and then attestation requests are sent to the host on which a VM instance is launched. Host checks the integrity of a VM image. The launching of a VM is decided based on VM image integrity. Tenant attested trusted cloud service [81] attests the VM image when it is received by a hypervisor. It uses Integrity Verification and Report Service (IVRS) at Dom0 to decrypt the VM image and derives the hash to verify its integrity. NIST [20] has recommended that the continuous monitoring of the VM states and in-out network traffic is necessary.

A summary of the hypervisor security solutions is presented in Table 6.

Table 6. Summary of Hypervisor Security Solutions
Approach Summary Limitations
Static and dynamic integrity measurement of hypervisor HyperGuard [86] protects the hypervisor code. HyperCheck [111] protects the integrity of hypervisor. HyperSentry[6] performs in-depth integrity measurement. HyperSafe [116] protects hypervisor control flow integrity. Technique [145] measures the checksum of the runtime hypervisor. MGUARD [63] monitors traffic between CPU and physical memory. HyperVerify [33] monitors hypervisor hardware state from Control VM. HyperGuard [86] secures only hypervisor code. HyperCheck [111] needs hardware changes. HyperSentry [6] cannot prevent the attack. HyperSafe [116] does not protect hypervisor non-control data. Technique [145] needs hardware changes. MGUARD [63] detects attacks based on only traces in physical memory. HyperVerify [33] faces overhead to translate low-level hardware state into hypervisor context.
Disaggregating the Management Interface; and Logging and auditing Disaggregating Xen [68] separates a VM-building functionality from Dom0. Nova[103] moves the hypervisor functionality to user level. Xoar [30] breaks the control VM into unprivileged service VMs. SSC [17] splits Dom0 into a systemwide Dom0 and a VM-specific Dom0. HypeBIOS [141] shifts Control VM from TCB. PSVM [136] separates Dom0 privileges. Nexen [97] provides multiple Dom0. TVGuarder [58] traces VM operations. In Reference [68], still attacks are possible through network drivers. Nova[103] cannot reduce the TCB completely. SSC [17] is still vulnerable through VM to hypervisor interaction. HypeBIOS [141] faces performance overhead for identifying crucial VMEXIT. PSVM[136] faces overhead of maintaining multiple Dom0s. Xoar [30] and Nexen [97] needs modification to Xen hypervisor. For Reference [58], it is difficult to identify illegal operations.
Minimizing the interfaces (e.g. Execution environment. Virtual resources, Physical resources) between hypervisor and VM sHype [89] controls resource sharing between VMs. NoHype [104] runs VMs natively at the top of physical resources. TrusrOSV [113] provides micro-hypervisor for each VM. Min-V [70] restricts re-engineering of boot time initialization code or earlier OS code. SplitVisor [73] provides a separate hypervisor to each VM. HyperLock [118] provides shadow hypervisor for each VM. DeHype [123] deprivileges the hypervisor execution. Nexen [97] provides VM specific Xen instance. sHype [89] cannot prevent external modification. In NoHype [104] and TrustOSV [113] dynamic resources allocation becomes difficult and still VMs interact with management OS. Min-V [70] Provides only boot time protection. In SplitVisor [73], new functionalities of hypervisor cannot be added dynamically. HyperLock [118] faces overhead of creating and pairing a hypervisor shadow to each VM. DeHype [123] faces overhead of identifying privileged operations. Nexen [97] needs modification to Xen hypervisor.
Hypervisor verification before initiating VM migration Role-based migration using Intel vPro TPM hardware [112]. TPM-based remote attestation [18]. Improved vTPM for property-based attestation [109]. Trust token-based VM migration protocol [3]. Technique [112] cannot be deployed in existing infrastructure. Remote attestation [18] does not provide confidentiality and integrity of data. Property-based attestation [109] increases migration time. In Reference [3], it is difficult to define trust level for each VM and hypervisor.
Verifying VM image integrity Cloud Verifier [92] and Tenant service [81] verify the VM image integrity. Cloud Verifier [92] needs to trust the middle agent with human interaction. In Tenant service [92], Dom0 can be compromised.

3.2 Virtual Machine Security

We discuss the existing security solutions in terms of securing running VM and VM image.

3.2.1 Security of VM in Running State. As shown in the threat model of the VM running state (refer Figure 4), VM can be attacked from different components of the virtualization.

Protecting VMs from the compromised/untrusted hypervisor. A tiny hypervisor-based approaches protect VMs from the compromised hypervisor. CloudVisor [138] separates resource management from security protection. A tiny security monitor provides security of the hosted VMs. Any access to VM contents from the hypervisor can be trapped. TinyChecker [107], a tiny nested hypervisor, protects guest VMs against hypervisor failure. It records all the communication between the VM and hypervisor. When any failure is detected, it provides an inplace recovery to preserve the VM state and data.

SICE [7] provides a hardware-level isolated and protected execution environment for the VMs on x86 multi-core architecture. It provides a hardware only TCB for VM. TCB includes the hardware, the BIOS, and the “System Management Mode (SMM).” A hypervisor-secure virtualization mechanism [105] uses the multi-core microprocessors to protect the VMs from the hypervisor. It focuses on VM memory protection during the execution. It stores cryptography keys in VM memory and controls that region of memory from the hypervisor or DMA access. HyperWall [106] implements hypervisor-secure virtualization using hardware to protect VMs from an untrusted hypervisor. Here, the hypervisor freely manages the VM resources such as memory, CPU cores, and so on. However, when VMs are created, the “Confidentiality and Integrity Protection (CIP)” tables prevent the hypervisor access to the memory of the guest VMs. HyperCoffer [128] protects the VM privacy and integrity from the untrusted hypervisor. It extends the processor virtualization to “memory encryption and integrity checking” for securing data communication. It introduces VM-Shim, which declassifies necessary information designated from VM to the hypervisor. H-SVM [51] protects the VMs from an untrusted hypervisor. It provides memory isolation by extending the current hardware support for memory virtualization using nested paging. It decouples memory isolation from memory allocation that results in reducing TCB to only the hardware.

SICE [7], a hypervisor-secure virtualization [105], HyperWall [106], HyperCoffer [128], and H-SVM [51] need to extend or modify existing hardware architecture. Similarly to CloudVisor, a hardware-assisted VM isolation called HA-VMSI [144] protects the guest VMs on ARM architecture during runtime. A thin security monitor separates memory isolation of VMs from the hypervisor. It uses hardware with the TrustZone support to protect the security monitor from the hypervisor and VMs.

A summary of the existing proposals for securing VM from the compromised hypervisor is given in Table 7.

Table 7. A Summary of the Existing Proposals for Securing VM from Compromised Hypervisor
Approach Summary Limitations
Tiny hypervisor-based protection CloudVisor [138] protects VMs from compromised hypervisor. TinyChecker [107] can recover VM after hypervisor failure. HA-VMSI [144] separates VM isolation functionalities from hypervisor. CloudVisor [138] cannot protect the security monitor. TinyChecker [107] considers only hardware, not the vulnerabilities. HA-VMSI [144] is dependent on hardware.
Hardware-based protection for VMs SICE [7] provides hardware-level isolated and protected VM execution. Hypervisor-secure virtualization [105] protects VM memory during execution. HyperWall [106] prevents access to VM memory. HyperCoffer [128] extends CPU virtualization to prevent memory access. H-SVM [51] provides hardware-based memory isolation. SICE [7] can be tampered by SMM rootkits. Hypervisor-secure virtualization [105] faces overhead due to dynamic management of physical resources and management functions can be abused. HyperWall [106], HyperCoffer [128] and H-SVM [51] need hardware modifications.

Protecting VMs from the compromised management interface/Malicious insider. Here, a secured hypervisor is considered. However, its management interface can be compromised by VMs or accessed by malicious insider/remote attacker.

The authors of Reference [56] have proposed a secure VM execution architecture to protect the guest VMs from an untrusted management VM. The management VM cannot access any memory of a VM after VM creation. It reduces the TCB of the guest VMs in an untrusted management environment. Here, DomU is protected from the Dom0, while allowing Dom0 to perform VM administrative tasks. Credo [78] protects guest VMs against a malicious administrator and malware in the root partition. It reduces the TCB of VM to a securely launched hypervisor and excludes the privileged partitions. It enables a VM to execute without taking any security dependency on the root partition. The authors of Reference [85] have proposed a memory access model for the management VM. It uses the principle of least privilege for management VM where it cannot read the memory allocated to the VM.

Although existing approaches [56, 78, 85] protect the VM memory from the management VM. They cannot restrict VM operations. The authors of Reference [129] prevent VM rollback attack by logging VM operations at a “small trusted computing base.” This log is audited to identify any unauthorized rollback, which constrains the VM operations. At starting of each operation, the VM hash is derived based on memory contents, its registers, and an image disk. The authors of Reference [122] have proposed a logging solution to produce the file-centric logs. It uses file-centric logs as evidence to enhance accountability of a VM. TVGuarder [58], a trace-enabled protection, detects illegal access of a VM. It is a log-based back-tracing mechanism that traces VM operations and allows only legitimate VM operations.

A summary of the existing proposals for securing VM from hypervisor management interface is given in Table 8.

Table 8. A Summary of the Existing Proposals for Securing VM from Hypervisor Management Interface
Approach Summary Limitations
Protecting VM rsources from management interface The authors of Reference [56] use hypervisor hardening to protect guest VMs from the control VM. Credo [78] provides VM execution independent to the root partition. A memory access model [85] restricts VM memory access The authors of Reference [56] require hardware changes to hypervisor. Credo [78] needs hardware changes. A memory access model [85] needs to differentiate between privileged and unprivileged memory access
Restricting VM operations. The authors of References [122, 129] Log and audit the VM operations. TVGuarder [58] traces insider's actions. The authors of Reference [129] focus only on malicious VM operations. The authors of Reference [122] log the file access only. TVGuarder's [58] deployment in virtual environment is a challenge.

Securing VM from kernel-level rootkits. Although protecting kernel code integrity is a primary step to maintain control-flow integrity and data integrity, they limit the further extension of kernel for better functionality. The DKOM- and DKSM-based rootkits can be detected based on data structure invariants, but any modification in data structure cannot be prevented. In addition, the return-oriented rootkits do not modify the kernel code but hijack the kernel control flow by modifying the existing kernel hooks. To protect such hooks, “Hooksafe” [117] relocates “kernel hooks” to the “page-aligned memory space” and then regulates accesses through “hardware-based page-level protection.” It protects hooks from rootkits by moving them to a write-protected area. HUKO [132] protects kernels from the untrusted kernel extensions. It allows execution of untrusted kernel extension and observes behavior through mandatory access control policies. It prevents rootkits through checking any violation in data and control flow integrity. Sentry [102] protects the “security-critical kernel data” used by the kernel. The kernel memory of guest OS is partitioned into different regions with access control for preventing the malicious modifications.

In VM Introspection (VMI) [69], the information delivered to hypervisor or derived at hypervisor is not enough to detect the kernel rootkits. Thus, researchers have focused on executing kernel identical to the target VM kernel for detecting hidden processes. Exterior [37] uses the writable VMI to execute an out-of-VM shell of an entire guest-OS. It uses a dual VM mechanism in which a secure VM (SVM) runs an identical kernel of the guest VM (GVM) and dynamically and transparently redirects the memory state at the hypervisor layer from SVM. Hybridbridge [87] uses data memorization and fall back at hypervisor.

Nitro [76] performs monitoring and analyzes the system calls to check the state of a VM from the hypervisor. NumChecker [114] detects the rootkits that are modifying control-flow. It uses Hardware Performance Counters (HPC) to count the number of hardware events that occur during the system call's execution. ShadowContext [126] reuses guest OS system calls in a shadowed portion of the out-of-VM inspection program to detect the rootkits. The authors of Reference [100] have collected the traces of HPCs for both malicious and trusted programs, and then machine-learning techniques are applied to detect rootkits. Such approaches do not need any pre-knowledge of the guest OS.

Protecting trusted applications from an untrusted kernel. To provide secure execution of the trusted application in the compromised kernel environment, it is necessary to protect application code and data from the kernel.

Hardware extensionsbased approaches like TrustVisor [65], Bastion [19], and SecureME [28] attempt to secure the execution of application in a malicious kernel environment. TrustVisor [65] offers code and data integrity as well as isolates the selected portions of an application using x86 hardware virtualization support. Bastion [19] enhances processor hardware by using a thin hypervisor and generates a security compartment for each application block. SecureME [28] secures an application from hardware attacks by using a secure processor. In addition, it protects the entire address space of an application from an untrusted OS through memory cloaking, permission paging, and system call protection.

Approaches like CHAOS [24], Overshadow [25], Virtual Ghost [31], and AppSec [82] attempt to enhance the hypervisor for securing application code and data in a malicious kernel environment. CHAOS [24] isolates CPU context and memory owned by a trusted process from the kernel and other processes. Overshadow [25] presents application resources as an encrypted view to the OS and thus allows the OS to manage these resources without read or modification operations. It uses multi-shadowing to present different views of physical memory. Virtual Ghost [31] combines compiler instrumentation and performs runtime checks on OS code to create ghost memory that the OS cannot read or write. AppSec [82], a hypervisor-based safe execution environment verifies the behavior of the untrusted OS according to the application's expectation. It uses a safe loader to verify the code integrity of the protected applications and avoids the compilation of applications statically.

InkTag [43] uses hypervisor with para-verification to monitor and verify the untrusted OS actions and offers process isolation. AppShield [26] protects the code, data, and runtime integrity of an application. It uses the Extend Page Table (EPT) provided by hardware virtualization extensions to isolate an application from the OS. MiniBox [57] protects an application from a malicious OS and secures a benign OS from a misbehaving application. Iso-X [34] provides isolated components to security-critical pieces of an application to securely execute an application on an untrusted OS. AppGuard [137] uses Intel's hardware virtualization support to protect an application. It uses a cryptographic approach only in system call-based I/O and paging-related I/O to ensure data privacy and integrity.

Haven [10] performs the shielded execution of an application in the presence of untrusted kernel. It leverages Intel SGX hardware to instantiate a secure region of address space known as an enclave. It prevents an access to enclave memory through hypervisor, BIOS, or OS. Glamdring [59] partitions the applications into security sensitive enclave and security-insensitive non-enclave parts. It protects the confidentiality and integrity of application data, even if an attacker has full control over the hypervisor.

A summary of the kernel-level attack detection and prevention techniques for VM is given in Table 9.

Table 9. Summary of the Proposals for Detecting and Preventing Kernel Level Attacks
Approach Summary Limitations
Protect kernel hooks from control-flow hijacking rootkits Hooksafe [117] protects kernel hooks by moving them to write protected area. Huko [132] protects kernels from untrusted kernel extensions. Sentry [102] avoids an illegal access to kernel data. Hooksafe [117] requires prior knowledge of kernel hooks to be protected. Huko [132] requires modification to the guest OS. Sentry [102] requires modification to the guest OS.
Identical kernel analysis Exterior [37] mimics the VM state in secure VM. Hybridbridge [87] performs data memorization and fall back at hypervisor. Nitro [126] runs system call code at “out-of-VM inspection program.” NumChecker [100, 114] detect rootkits based on hardware performance counters. For Exterior [37], it is difficult to maintain copies of all VMs. Hybridbridge [87] faces high performance overhead. Nitro [126] faces performance overhead. NumChecker [114] cannot determine the most significant counters. Technique [100] needs to be trained for normal applications and malware.
Hardware extensions-based application protection TrustVisor [65] isolates the application block. Bastion [19] generates a security compartment for application block. SecureMe [28] uses memory cloaking, permission paging and system call protection. CHAOS [24] isolates CPU context and memory of application. Overshadow [25] presents application resources as an encrypted view to the kernel. Virtual Ghost [31] creates ghost memory for an application. AppSec [82] protects the application code. TrustVisor [65] cannot protect whole application. Bastion [19] is too restrictive for protecting user data or an entire application. SecureMe [28] requires modifications to the OS and applications. CHAOS [24] needs modification to the OS and code expansion to the hypervisor. Overshadow [25] needs complex encryption and decryption operations. Virtual Ghost [31] needs to modify or re-compile the OS kernel. AppSec [82] cannot verify the correctness of kernel.
Hypervisor-based protection of an application from Iago attack InkTag [43] monitors the untrusted kernel actions. AppShield [26] uses hardware virtualization extensions to isolate an application. MiniBox [57] protects an application from a malicious kernel. Iso-X [34] isolates the execution of critical part of an application. AppGuard [137] uses cryptographic approach for system call and paging I/O. InkTag [43] needs to modify the source code of the kernel. AppShield [26] needs a trusted component in the guest kernel. MiniBox [57] cannot protect whole application. Iso-X [34] cannot protect whole application. AppGuard [137] adds a small amount of performance overhead.
Protect applications from kernel/HMI Haven [10] and Glamdring [59] use Intel SGX enclave to protect the security critical application code. In Haven [10], placing all code inside the enclave increases the TCB. In Glamdring [59], security code may access sensitive data.

Protecting VMs from the rouge co-hosted VM. Many proposals are attempting hard isolation, which dedicates separate hardware for each VM. However, hard isolation leads to reduced efficiency. The authors of Reference [77] have prevented cache sharing among the VMs. The authors of Reference [96] have designed a dynamic page coloring for cache isolation during security-critical operations. Page coloring uses several colors for secure processes, and these colors cannot be used by other VMs during the execution of security critical operations. The authors of Reference [54] have proposed a page coloring to isolate cache lines in Xen. For each VM, it locks/reserves the cache lines of each CPU core where security-sensitive code and data are stored. The locked lines are not allowed to read by other VMs, and, thus, a VM can load sensitive data into the locked cache lines. The authors of Reference [38] have partitioned the cache memory during boot time. At boot time, a VM belonging to a particular partition is only permitted to use memory of the same partition and, thus, VM is confined to a fixed partition of the shared cache. SAFEPERIMETER [44] secures an access of LLC cache by locking it line by line without assigning any locked cache line to other VMs. The authors of Reference [108] have reduced the risk of resource sharing through better scheduling. Here, the Xen scheduler is modified to limit the frequency in which an attacker can preempt the victim. Nomad [66] identifies a co-residency as a root cause of a side-channel attack and prevents the information leakage by migrating the target VM. Due to migration, the VM co-location becomes difficult for an attacker. CATalyst [61] prevents LLC side-channel attacks using Intel's performance optimization feature called Cache Allocation Technology (CAT). It partitions the LLC into secure and non-secure partitions. The non-secure partition is hardware managed and the secure partition is software managed. The secure partition uses CAT to enforce the cache-pinning of secure pages. It uses secure partition to store sensitive code and data so that malicious VMs cannot read the sensitive data. The authors of Reference [143] have proposed a software-only defense against attacks like Flush+Reload and Prime+Probe. It uses a copy-on-access mechanism to create a separate copy of physical pages when accessed by other VMs. Thus, the access of VM to its own copy is invisible to an attacker in a Flush+Reload attack. To defeat Prime+Probe attacks, it controls the cacheability of memory pages so that the memory of VM mapped to cache sets becomes invisible to the attacker. The authors of Reference [101] have eliminated LLC-based side-channel attacks using the combination of CAT and Completely-Fair-Scheduler. It dynamically schedules the cache partition that belongs to a separate VM.

Observer [60] runs a secure VM that mimics the vulnerable VM and redirects all inbound traffic destined to the vulnerable VM to the secure VM. It differentiates the outbound traffic to detect covert channels. BusMonitor, a hypervisor-based protection system [90], prevents misuse of memory bus operation. It partitions the memory bus operations among VMs to prevent a cross-VM side channel. C2Detector [125] consists of captor and detector. Captor captures hypercalls generated by VMs. Detector at Dom0 analyzes the captured hypercall records and detects the covert channels.

ANVIL [5] is designed to detect RowHammer attack. It tracks the locality of DRAM accesses using existing hardware performance counters. It identifies the rows being frequently accessed and then selectively refreshes the nearby rows to prevent hammering. B-CATT [14] extends the system bootloader to disable vulnerable physical memory. It identifies vulnerable rows and prevents RowHammer attack. CATT [13] divides the physical memory into multiple partitions where each partition is separated by at least one DRAM row and owned by a specified domain. Thus, bit-flips induced by one domain do not affect the memory partition of other domains. It is implemented for user and kernel domains.

The existing proposals for securing VM from co-hosted VMs are summarized in Table 10.

Table 10. A Summary of the Existing Proposals for Securing VM from the Co-hosted VMs
Attack Summary Limitations
Side-channel attack Techniques such as Cache partitioning [77], Dynamic page coloring [96], Page coloring and locking [54], Boot time memory partitioning [38], Reduced resource sharing through scheduling [108], Migrating the target VM [66], Line by line cache locking [44], Dynamic partitioning of LLC using CAT [61], Copy-on-access [143], Dynamic partitioning and scheduling of LLC partitions [101] protect VM resources from co-hosted VM. Cache partitioning [77] faces overhead on resource management and requires hardware changes. Dynamic page coloring [96] requires frequent swapping pages between colors. Page coloring and locking [54] cannot detect LLC-based covert channels. Boot time memory partitioning [38] limits the resource utilization. Reduced resource sharing [108] needs modification to the Xen scheduler. Migrating VM [66] is not time efficient. Line by line cache locking [44] adds overhead on resource management. Dynamic partitioning [61] needs to determine data and instructions that are to be executed in secure partition. Copy-on-access [143] cannot maintain multiple copies of accessed memory region. Scheduling of LLC partitions [101] is not feasible with recent hypervisors.
Covert-channel attack Observer [60] mimics the vulnerable VM to detect covert channels. BusMonitor [90] partitions bus operations between VMs. C2Detector [125] detects OS-level covert-channel attack. For Observer [60], maintaining a secure copy of every VM is a complex process. BusMonitor [90] increases performance overhead. In C2Detector [125], switching between secure VM and hypervisor increases the performance overhead.
Row- Hammer attack ANVIL [5] refreshes the adjacent rows. B-CATT [14] disables vulnerable rows. CATT [13] separates physical memory partitions by at least one DRAM row. ANVIL [5] induces a worst-case overhead of 8% and suffers from false positives. B-CATT [14] does not tackle the memory isolation in physical memory. CATT [13] needs to be extended for VM domains.

3.2.2 Security of VM in Inactive State. As per NIST, common recommendations for the VM image security are to use an encrypted and digitally signed images and place the VM image repository outside the hypervisor with the strict access control.

Securing image from the outdated software and known vulnerabilities. Mirage [120] periodically executes malware detection tools and discovers the vulnerabilities for patching. The authors of Reference [142] have used Mirage image library to patch multiple images offline. It uses a script rewriting technique to create safe patch scripts. Similarly, authors [35] have considered two modules: collector and patcher. The collector identifies the outdated software and malware. The patcher applies patches to the identified vulnerabilities. When a VM image is launched to the hypervisor, the outdated software and VM software vulnerabilities can be identified for further patching [93]. ImageElves [50] identifies the running VMs that need to be patched and creates reliable patches using online updates. It groups the VMs into equivalent classes and applies update to one VM and the update image is generated for other VMs.

Securing VM image in repository. “Mirage image format” [80] gives the semantic information that is hidden in the disk-image files. For each image, it maintains a file name for mapping it to the semantic information. Thus, it restricts the creation of multiple images. Mirage [120] protects the images from an unauthorized access using check-in- and check-out-based access control. A tracking mechanism keeps track of all the VM images. An access control mechanism [36] prevents the placement of the poisoned images or modification of images in a repository using a role-based access control mechanism. It applies the authorization rights to the user and logs its repository accesses. “Encrypted Virtual Disk Images in Cloud (EVDIC)” [53] protects the stored VM images using an advanced encryption standard. While loading the image, it is decrypted by interacting with the key distribution server. It protects VM image confidentiality and integrity.

Securing VM image from MITM attack and destination hypervisor. A trusted cloud computing platform (TCCP) [91] has two elements: “Trusted Virtual Machine Monitors (TVMM)” and a “Trusted Coordinator (TC).” TC is hosted at an external trusted entity and manages the set of the trusted hypervisors. Each hypervisor runs a TVMM that is verified by TC. A VM user cooperates with TC to launch the VM image to the trusted hypervisor. Similarly, Cloud Verifier [92] verifies the integrity of VM and hypervisor and helps VM user to launch their VM image to a trusted hypervisor. In TCCP [91] and Cloud Verifier [92], the coordinator needs to be trusted to verify the trustworthiness of the hypervisor. A secure VM launch protocol [4] avoids the middle coordinator and uses the combination of IPsec, an asymmetric key cryptosystem, and trusted computing techniques to enable a VM user to bind a VM to a hypervisor that has been booted into a trustworthy state. Similarly, the authors of Reference [72] have ensured the launch of a VM image to a trusted hypervisor using the trusted third party (TTP). A mobile agent-based secure launch protocol [74] protects the VM images against an MITM attack using end-to-end symmetric key encryption. In addition, the mobile agent is encapsulated with a VM image that verifies the trustworthiness of the hypervisor and calculates the decryption key to decrypt the image. A tenant-attested trusted cloud service [81] allows a VM owner to check whether the Dom0 is trusted or not.

Securing VM image from data remanence attack. The authors of Reference [1] have studied different approaches to defend against data remanence attack. It includes sanitization [21], encryption, and so on. In sanitization [21], the hardware assigned to VM can be overwritten with new data. In addition, VM sensitive information can be encrypted and keys can be stored on a VM.

Table 11 presents a summary of the existing proposals to secure VM images.

Table 11. A Summary of the Existing Proposals to Secure VM Images
VM image location Summary Limitations
Destination hypervisor Mirage [120] periodically executes malware detection tools. The authors of Reference [142] patch images offline. The authors of Reference [93] patch old software and known vulnerabilities. Technique [35] patches VM images offline. ImageElves [50] patches VM image through online updates. Filters in Mirage [120] cannot scan all malwares. In Reference [142], suspended VM cannot be patched. The authors of Reference [93] require manual process. Technique [35] work only on dormant images. In ImageElves [50], patch-based image classification is very difficult.
Image repository Mirage image format [80] protects images sprawling. Mirage [120] filters poisoned images. Access control mechanism [36] prevents the placement of poisoned images or modifying existing images. EVDIC [53] uses symmetric key encryption. Mirage image format [80] cannot protect an illegal access to VM images. In Mirage [120], access by filters raises the image privacy issue. Access control mechanism [36] cannot maintain the roles in virtualization hierarchy. In EVDIC[53], encryption/decryption cost is high when images are more.
Launching channel and Destination hypervisor. TCCP [91] and Cloud Verifier [92] guarantee the launching of VM to trusted hypervisor. The authors of Reference [4] allow VM user to verify the hypervisor. The authors of Reference [72] use a TTP to VM image launch. The authors of Reference [74] provide a mobile agent-based secure image launch protocol. Tenant attested service [81] verifies integrity of Dom0. TCCP [91] is difficult to adopt because of an external trusted entity. Cloud Verifier [92] needs regular human interaction. The authors of Reference [4] limit the flexibility of load balancing. In Reference [72], trustworthiness of hypervisor is not consistent. In Reference [74], key distribution is a great challenge. Tenant attested service [81] faces performance overhead.
Physical hardware Sanitization [21] overwrites VM data. Enxryption [1] encrypts VM data. Sanitization [21] is time consuming. In Enxryption [1], key management is difficult.

3.3 Virtual Network Security

For virtual network security, the security solutions such as Firewall, NIDS, NIPS, DIDS, and so on, can be used. However, these solutions face the problem of dynamic network confugirations, lack of centralized control, and high detection latency. Thus, researchers attempt to focus on SDN-based security for virtual networks. SDN enhances network security through the features such as centralized control, dynamic network configuration, network traffic control, network programmability, and so on.

CLOUDWATCHER [98] dynamically changes routing paths of the packets for inspection at pre-installed network security devices. NETSECVISOR [99] uses the existing network security devices and leverages SDN to redirect the network traffic to security devices. NetFuse [115] protects traffic overload in SDN-based data center networks. Network intrusion detection and the countermeasure sElection (NICE) [29] framework deploys NICE agent at each physical server for network traffic analysis. It assesses VM network vulnerabilities to generate an attack graph and then, as per impact of vulnerability, it applies “deep packet inspection (DPI)” and/or virtual network reconfiguration. DCPortalsNg [67] isolates virtual network traffic using SDN. It unpacks the network packet and replaces the source and destination IP addresses. In addition, the MAC address is replaced with host's MAC address. The packet rewriting guarantees that traffic from a tenant will be out of reach for other tenants. The authors of Reference [110] have used SDN for DDoS attack mitigation. It monitors the global view of network and quickly builds the control structure for fast response to attack. The authors of Reference [88] attempted to find a DDoS attack class and make the controller install the corresponding rule in the OpenFlow switch. FlowTrApp [15] uses network traffic flow rate and duration to detect DDoS attack ranging from low rate to high rate and long lived to short lived attacks.


We discuss the research scope in terms of security requirements for each component of the virtualization.

4.1 Hypervisor Security Requirements

Although there have been several works for hypervisor security, still there is a scope for further improvements.

4.1.1 Protecting Hypervisor Integrity.

Protecting integrity measurement agent. The hypervisor integrity protection mechanisms need a trusted component that can guarantee the security of the integrity measurement agent. HyperGuard [86] and HyperCheck [111] use hardware (e.g., SMM), HyperVerify [33] uses a control VM or HyperSentry [6], and MGUARD [63] use a combination of hardware and software as a trusted component. One of the issues with this approach is that these approaches consider only a specific threat model. However, with advances in technology, new threats may emerge, for example, SMM used in HyperGuard [86] and HyperCheck [111] needs to be considered for hardware-based rootkits. In addition, HyperVerify [33] needs to be considered for a control VM compromise.

Complete hypervisor introspection. As HyperGuard [86], HyperCheck [111], and HyperSentry [6] provide static hypervisor code integrity, they can be bypassed by the return-oriented rootkits. HyperSafe [116] protects the control flow integrity. The authors of Reference [145] verify only the checksum of hypervisor code. However, they cannot protect hypervisor non-control data. HyperVerify [33] protects hypervisor non-control data. It faces overhead when deriving high-level hypervisor context from the low-level hardware state.

4.1.2 Protecting Hypervisor from the Management Interface/Malicious Insider. When securing hypervisor from the management interface, researchers either disaggregate the management interface or restrict access to hypervisor functionalities.

Reducing the attack surface through disaggregation. Disaggregated Xen [68], Nova [103], Xoar [30], SSC [17], and PSVM [136] have minimized the interaction between the hypervisor and management interface by taking Dom0 away from the TCB of the hypervisor. However, still they have left the large TCB of the hypervisor for VM to VM or VM to hypervisor interaction. Thus, hypervisor remains vulnerable to attacks through other exposed interfaces like network drivers, block drivers, and so on. HyprBIOS [141] have attempted to reduce the VM to VM and VM to hypervisor interaction based on access control violation by VMEXIT. However, identifying the VMEXIT of interest adds the performance overhead. In addition, it has left a surface for DOS attack from VM to VM. Recently, Nexen [97] have attempted to decompose Xen into multiple VM-specific Xen instances for providing an isolated Xen environment to each VM. However, they require new design of the Xen hypervisor. They have not considered the attacks from the malicious insider or attacks during the live migration of the VM. For example, PV guests are typically run in shadow mode for live migration, as well as for features like VM snapshot. In this case, PV guest can crash the hypervisor (CVE-2017-17565). Nexen only controls the memory accesses; however, other hardware resources are shared by VMs and Xen slices. It is again an attack surface that, if compromised, may crash the hypervisor through the DoS attack (CVE-2015-4105). Nexen can be vulnerable to Iago attack where VM kernel is malicious and may attack the VM applications. The shared service domain performs the functionalities that need to cross the Xen slice boundary. In addition, Nexen cannot handle hardware bugs (CVE-2017-15594) (CVE-2017-10916).

Restricting illegal VM operations. Although disaggregation helps in deprivileging the management interface, the access to management interface by malicious insider or remote attacker also needs to be considered. Usually VM rollback and VM sprawl attacks are emerged from a poor VM management. TVGuarder [58] allows only genuine VM operations using logging and auditing. However, in a virtual environment where VMs need to be operated frequently, differentiating illegal and genuine VM operations is a challenge. It may face the overhead of the false-positive rate.

4.1.3 Protecting Hypervisor from Rouge VMs. Hypervisor can be secured from VMs by provisioning either a strong isolation to hypervisor resources from VMs or an isolated execution environment for VM.

Protecting hypervisor resources from VMs. The approaches like sHype [89], NoHype [104], Min-V [70], and TrustOSV [113] protect hypervisor resources through controlled sharing. However, they limit dynamic resource allocation, which is a vital need of virtualization-based computing technologies. In addition, still the hypervisor shares some resources with the VM through which the guest can crash the hypervisor.

Providing isolated execution environment. HyperLock [118], DeHype [123], and SplitVisor [73] attempted to provide a separate hypervisor instance for each VM. Similarly, Nexen [97] decomposes Xen to provide a separate Xen instance for each VM. However, this design is not backward-compatible with commercial hypervisors and, thus, existing computing models need to be migrated with heavy cost. In addition, creating a separate hypervisor instance in a dynamic virtualization environment is an extra overhead, which may degrade the hypervisor performance. Although this design is adopted in future computing models, still it poses security concerns related to the monitoring code that runs in the privileged mode, leaving an additional attack surface.

4.1.4 Verifying Trustworthiness of Destination Hypervisor. The authors of References [3, 18, 109, 112] have used trusted computing technologies (e.g., TPM) to perform attestation of the hypervisor where VM is to be migrated. However, in practice, trusted computing technologies use cryptography, and, thus, distributing and managing keys in dynamic virtual environment is very difficult. The deployment of trusted computing technologies needs TPM-compatible hardware.

4.1.5 Protecting Hypervisor from VM User. Cloud Verifier [92] and Tenant-attested trusted cloud service [81] verify the integrity of a VM image that is to be launched. One of the issues is that these approaches need to maintain hypervisor or secure VM integrity. They do not consider any compromise of these components from a VM user.

4.2 VM Running State Security Requirements

4.2.1 Protecting VMs from the Compromised Hypervisor. To protect VM from the compromised hypervisor, existing solutions use a trusted tiny hypervisor or hardware-based protection.

Protecting a security monitor. CloudVisor [138] and TinyChecker [107] use a tiny nested hypervisor as a security monitor to protect VMs from the hypervisor. However, these approaches do not consider the integrity of a security monitor, which may result in a single point of failure. In addition, they do not defend against physical attacks. HA-VMSI [144] protects a security monitor using a trusted hardware component (TrustZone). However, while using hardware as a trusted component, its security needs to be considered against hardware rootkits and physical attacks. In addition, as it is only implemented on ARM architecture, its feasibility on x86 architecture needs to be checked. In future, it can be implemented on x86 architecture with the Intel SGX support where security monitor can be placed in SGX enclave.

Protecting VM resources from hypervisor. The hardware-based approaches like SICE [7] a hypervisor-secure virtualization [105], HyperWall [106], HyperCoffer [128], and H-SVM [51] protect the VM memory from hypervisor during execution. These approaches do not consider any illegal access from a management interface that is involved in performing privileged operations (e.g. VM snapshot, rollback and so on) and may need access to the VM memory.

4.2.2 Protecting VM from Management Interface.

Protecting VM resources from management interface. The authors of References [56, 78, 85] protect VM memory access from the management interface. They cannot restrict illegal operations on VM by malicious insider. However, through specific VM operations such as cloning or relocation, it is possible to affect the VM privacy without accessing the VM memory.

Restricting illegal VM operations. The authors of References [58, 122, 129] restrict the the illegal VM operations. However, differentiating between privileged and unprivileged VM operations is a challenge. In addition, their deployment should be out of reach to the insider.

4.2.3 Protecting VM Kernel. While protecting the VM, it should be viewed through its two components, kernel and trusted applications.

Kernel hook protection. Hooksafe [117], HUKO [132], and Sentry [102] protect the kernel hooks of the original kernel from being hijacked. However, identifying the kernel hooks of the rootkit interest is a challenge at the hypervisor level and causes extra overhead.

Complete introspection (Leveraging Inspection). Exterior [37] and Hybridbridge [87] perform cross-verification of VM kernel state to detect rootkits. Although identical kernel state is observed, the complete view of the VM cannot be available. These approaches perform kernel memory introspection only. However, some rootkits can perform repeated logging to the hypervisor and fill up the disk, causing denial of service to the hypervisor. Thus, the (log) files on the disk also need to be introspected.

Protecting security solution (Leveraging Isolation). The rootkit detection and prevention systems like Exterior [37] and Hybridbridge [87] use secure VM as a part of their security solution. However, secure VM has different interfaces through which it can be compromised. In addition, an attacker having an access to secure VM can tamper with the security solution.

In-place prevention (Leveraging Interposition). Hooksafe [117], HUKO [132], and Sentry [102] protect the original kernel hooks. However, it is necessary to provide in-place prevention of rootkits. Using writable VMI, the immediate response to rootkits can be given. It includes killing a rootkit, restricting write access to rootkits, and so on.

Minimum overhead. Exterior [37] and Hybridbridge [87] mimic the VM kernel state in secure VM to get the complete view of the VM kernel. These systems are designed by considering single-VM for the scan. However, they face the problem of performance overhead if there are multiple VMs to be scanned for the rootkits. The hardware-based lightweight approaches [100, 114, 126] detect rootkits through system call interception; however, they cannot prevent the rootkits. In addition, these approaches need to be trained for the normal behavior of the kernel and newer rootkits.

Early detection. In rootkit detection, the target VM gets affected at least once that may result in affecting the CIA of whole virtual environment. In addition, the rootkit prevention systems do not allow unauthorized kernel code executions, which limits the execution of some useful kernel extensions. Thus, in future, early detection and prevention can be performed on the VM immediately when it joins the virtual environment. Here, VM needs to be isolated from an entire virtual environment with the extra cost of delay to start the VM services.

4.2.4 Providing an Isolated Execution Environment for Applications. In a virtual environment, an application needs to be protected from untrusted kernel, hypervisor, and physical access. Hardware-based approaches like TrustVisor [65], Bastion [19], and SecureME [28] and hypervisor-based approaches like CHAOS [24], Overshadow [25], Virtual Ghost [31], and AppSec [82] isolate the memory and CPU state of an application. However, these approaches are vulnerable to Iago attack through system call interface. The approaches like InkTag [43], AppShield [26], MiniBox [57], Iso-X [34], and AppGuard [137] assume a trusted hypervisor and cannot protect applications from an attacker having control over the hypervisor. In addition, they add extra functionalities such as memory access control, isolation, encryption, and so on, to the hypervisor, which increases the size of the hypervisor TCB, resulting in a new attack surface. Haven [10] and Glamdring [59] protect the applications without trusting the hypervisor or any hardware beyond the processor. However, they place the application code in the Intel SGX enclave where code executes at a higher privilege level with access to sensitive data. These approaches focus on security of only single-VM, and thus infeasible to directly apply for securing multi-tenant VMs.

4.2.5 Secure Resource Sharing. Different techniques have been used for restricting illegal access through the shared resources. The authors of Reference [77] use static cache partitioning. The authors of Reference [96] use dynamic page coloring. The authors of Reference [54] use page coloring and locking. The authors of Reference [38] use boot time cache partitioning. SAFEPERIMETER [44] uses line-by-line cache locking. However, these techniques limit resource utilization, which is a vital feature of virtualization-based computing technologies. The authors of Reference [108] use scheduling-based resource sharing to improve resource utilization. However, it needs modification of the original Xen scheduler. Nomad [66] limits co-residency through VM migration. However, it is not efficient to avoid side channel. CATalyst [61] uses CAT to store sensitive data and instructions in a secure partition. However, it needs to determine the instructions and data to be executed in a secure partition. The authors of Reference [143] use a copy-on-access, where it is difficult to maintain multiple copies of the accessed memory regions. The authors of Reference [101] use the combination of CAT and Completely-Fair-Scheduler. However, it is not compatible with all well-known hypervisors since it needs a hypervisor-specific scheduler.

To detect covert channels, Observer [60] uses outbound traffic in SVM, BusMonitor [90] partitions the memory bus operations, and C2Detector [125] uses hypercalls. However, these approaches increase the performance overhead on the hypervisor in theh case of multiple target VMs.

ANVIL [5] refreshes the adjacent rows, and B-CATT [14] disables the vulnerable rows to prevent a RowHammer attack. However, identifying the rows under attack is a challenge, since some benign applications may access the rows frequently. In addition, it adds extra overhead. CATT [13] partitions the physical memory, where each partition is separated by at least one DRAM row. However, it restricts the dynamic memory allocation.

4.3 VM Inactive State Security Requirements

Although there have been several approaches proposed for VM image security, there is room to enhance the security of a VM image.

4.3.1 Keeping VM Image Updated. When performing patching of VM images, it is important to determine when it needs to be patched. It is emphasized that to apply patches, the following steps need to be considered [35]: (1) Identify VM to be updated, (2) identify outdated software and known vulnerabilities, and (3) create patches from online updates and apply them to VM. The authors of References [50, 93] have followed the necessary steps to perform online patching of image when it is launched to the hypervisor. However, it is not efficient if there are a higher number of images to be patched at a time. The best practice is to maintain a patch repository along with image repository and keep it updated with recent patches of well-known OSes. When a VM image is created or deployed in the hypervisor, immediately it should be patched with recent patches from the patch repository.

4.3.2 Securing Image in Repository. To combat the threat against a VM image in a repository, either image read can be restricted or the access control policies can be improved.

Restrict the image read. Researchers are adopting an encryption-based restriction on images. For instance, the authors of Reference [53] have encrypted images with symmetric key encryption. However, a VM image needs to be decrypted before its use on the hypervisor. This may degrade hypervisor performance. In addition, it cannot limit the attacker to tamper or destroy the images in the repository.

Improving access control policies. When improving the access control, it is important to determine what all interfaces have access to in the repository, as the repository has many interfaces other than the image publisher and retriever. The authors of Reference [120] have considered access control for the image publisher and retriever. It does not consider access control over the image filters. The authors of Reference [36] have used role-based access control. However, a repository may have an increasing number of users, and the administrator needs to create corresponding roles. Managing all these roles become complex. In addition, the authors of References [36, 120] do not consider access control over the patching interface, which may risk image privacy. Thus, research needs to be done to improve access control. In future, attribute-based access control can be applied, which is an advanced way of handling authorization.

4.3.3 Securing Image in Transit. To protect a image on channel, researchers have adopted end-to-end security from the repository to the hypervisor. Recently, the authors of Reference [74] have proposed a security protocol where symmetric key encryption is used to protect image confidentiality, and hashing is used to protect image integrity. As the decryption key needs to be derived by the hypervisor, it also faces the problem of hypervisor performance degradation.

4.3.4 Verifying Trustworthiness of Hypervisor. To verify the trustworthiness of the hypervisor, existing solutions use either trusted computing technologies [72, 91, 92], direct verification by a VM user [4], or mobile agent-based hypervisor verification [74]. However, TTP needs to be trusted to perform the actual verification of the target hypervisor. The direct communication between the VM user and the hypervisor can be tampered with. The authors of Reference [74] have faced performance overhead when encrypting and decrypting a VM image.

4.3.5 Destroying VM Data. The physical destruction of VM data is very difficult in virtualization. In sanitization [21], overwriting the VM hardware is time consuming. Encryption makes VM data invisible; however, data processing in virtualization becomes difficult. The authors of Reference [1] have suggested to combine both techniques to sanitize only hardware area that contains keys. However, identifying the hardware area that contains the keys is an extra overhead.

Table 12 presents the summary of the hypervisor and running VM security by considering defense techniques against attacks.

Table 12. A Summary of the Virtualization Security Approaches
Objective/Security solutions Attack index as per Figure 2
1 2 3 4 5 6 7 8 9
Hyp. static integrity check [6, 60, 63, 86, 111] P P - - - P - - - - - -
Hyp. runtime integrity check [33, 116, 145] - - - P - - - - - -
Disaggregate the HMI [17, 30, 68, 103] P P P P P P - - P P - - -
Disaggregate the HMI [141] P P - - P P - - -
Disaggregate the HMI [136] P P P P P P - - P P - - -
Disaggregate the HMI [97] P P P - - P P P - - -
Restrict hyp. operations [58] - - - - - - - - - - - - -
Restrict access to hyp. resources [70, 73, 89, 104, 113] P P - - P - - - - - -
Providing isolated hyp. [118, 123] P - - - - P - - -
Providing isolated hyp. [97] P P P - - P P P - - -
Securing hyp. from network [3, 18, 109, 112] P - - P - - - - - - - - -
Securing hyp. from VM user [81, 92] P P P P P P P P P P - - - - - - - - -
Protect VMs from compromised hyp. [7, 51, 105, 106, 107, 128, 138, 144] - - - P P - - -
Protect VM resources from CHMI [56, 78, 85] - P - - P - - -
Restrict operations on VM [58, 122, 129] - - - - - - - - - - P - - - -
Detect/prevent kernel rootkits [37, 87, 100, 102, 114, 117, 126, 132] - - - - - - - - - - - - - - - - - -
Protect applications from untrusted kernel and hyp. [19, 24, 25, 28, 31, 65, 82] - - - - - - - - - - - - P P - - -
Protect applications from untrusted kernel and hyp. [26, 34, 43, 57, 137] - - - - - - - - - - - - P - - -
Protect applications from untrusted kernel and hyp. [10, 59] - - - - - - - - - - - - P P P P - - -
Prevent side-channel attacks [38, 44, 54, 61, 66, 77, 96, 101, 108, 143] - - - - - - - - - - -
Prevent covert-channel attacks [60, 90, 125] - - - - - - - - - - -
Prevent RowHammer attacks [5, 13, 14] - - - - - - - - - - - P P

Hyp.: Hypervisor, : Fulfilled, ✗: Not fulfilled, P: Partially fulfilled, -:Not considered.

4.4 Virtual Network Security Requirements

In addition to the basic virtual network security requirements such as detecting a variety of attacks, minimizing communication overhead, resistance to compromise, and so on, providing SDN security is one of the crucial requirements that is raised with the use of SDN in virtual network security. SDN secures the virtual network through IDS, IPS, DIDS, and so on. However, it reveals a technique to control and operate the virtual network, which can be targeted by an attacker. If communication from the control plane to data planes is not properly secured, it may affect the security of an entire network, and therefore the SDN control plane and related access control mechanisms should be properly designed and implemented.


Virtualization is considered as a base technology in the construct of modern computing technologies. However, due to vulnerabilities existing in current implementations of virtualization platforms, the demand for secure virtualization is increased. We have attempted to show various vulnerabilities, threats, and attacks at different components of virtualization, which inherently pose several security and privacy concerns in modern computing technologies. By investigating the existing security solutions, we have identified the research scope and some future challenges. There is a need of an advanced security model and better access control and cryptoalgorithms that can target different levels of security for each component. Our findings can help the research community address virtualization security challenges and to make virtualization a platform for the delivery of secured and trusted computing technologies.



This work is a part of the project titled “Designing out-of-VM Monitoring based Virtual Machine Introspection Framework for Securing Virtual Environment of Cloud Computing (ECR/2017/001221)” with funding support from Science and Engineering Research Board, Department of Science and Technology, Government of India.

Authors’ address: R. Patil and C. Modi, National Institute of Technology Goa, Ponda, Goa 403401, India, emails:,

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from

©2019 Copyright held by the owner/author(s). Publication rights licensed to ACM.
0360-0300/2019/01-ART12 $15.00

Publication History: Received July 2017; revised September 2018; accepted October 2018