VirtAV: An agentless antivirus system based on in-memory signature scanning for virtual machine

Antivirus is an important issue to the security of virtual machine (VM). According to where the antivirus system resides, the existing approaches can be categorized into three classes: internal approach, external approach and hybrid approach. However, for the internal approach, it is susceptible to attacks and may cause antivirus storm and rollback vulnerability problems. On the other hand, for the external approach, the antivirus systems built upon VMI technology cannot find and prohibit viruses promptly. Although the hybrid approach performs virus scanning out of the virtual machine, it is still vulnerable to attacks since it completely depends on the agent and hooks to deliver events in the guest operating system. To solve the aforementioned problems, based on in-memory signature scanning, we propose an agentless antivirus system VirtAV. VirtAV can monitor the specific event of the guest VM that is defined as the first instruction-fetch operation on a newly updated host memory page frame, and can scan virus in the page when the event occurs. As an external approach, VirtAV doesn't rely on any event or agent in the guest OS, so it guarantees the security of itself to the greatest extent. In addition, it provides full life cycle protection for VMs, no matter which state (running, paused, resumed or migrated) they are in. We implemented a prototype by extending Qemu/KVM hypervisor. Experimental result demonstrates that the function of VirtAV is verified (by finding 100% of the 3546 sample viruses) and the overhead of VirtAV on guest performance is acceptable. Especially, VirtAV has little impact on the performance of common desktop applications, such as video playing, web browsing and Microsoft Office series.