Finding Advanced Malware Using Volatility

Hakin9 Magazine now has a course on Volatility, you should check it out ->


by Monnappa Ka

What you should know

  • Basic understanding of malware

  • Knowledge of operating system processes

  • Understanding of Windows Internals

What you will learn

  • Performing memory forensics

  • Tools and techniques to detect advanced malware using Memory forensics

  • Volatility usage

Memory Forensics is the analysis of the memory image taken from the running computer. Memory forensics plays an important role in investigations and incident response. It can help in extracting forensics artifacts from a computer's memory like running process, network connections, loaded modules etc. It can also help in unpacking, Rootkit detection and reverse engineering. When an organization is a victim of advanced malware infection, a quick response action is required to identify the indicators associated with that malware to remediate, establish better security controls and to prevent future ones from occurring. In this article you will learn to detect advance malware infection in memory using a technique called "Memory Forensics" and you will also learn to use Memory Forensic Toolkits such as Volatility to detect advanced malware with a real case scenario.

Steps in memory Forensics

Below is the list of steps involved in memory forensics:

a) Memory Acquisition - This step involves dumping the memory of the target machine. On the physical machine you can use tools like Win32dd/Win64dd, Memoryze, DumpIt, FastDump. Whereas on the virtual machine, acquiring the memory image is easy, you can do it by suspending the VM and grabbing the ".vmem" file.

b) Memory Analysis - once a memory image is acquired, the next step is to analyze the grabbed memory dump for forensic artifacts, tools like Volatility and others like Memoryze can be used to analyze the memory

Volatility quick overview

Volatility is an advanced memory forensic framework written in python. Once the memory image has been acquired Volatility framework can be used to perform memory forensics on the acquired memory image. Volatility can be installed on multiple operating systems (Windows, Linux, Mac OS X), Installation details of volatility can be found at

Volatility Syntax

  • Using -h or --help option will display help options and list of a available plugins

    python -h

  • Use -f <filename>  and --profile to indicate the memory dump you are analyzing

    python -f mem.dmp --profile=WinXPSP3x86

  • To know the --profile  info use below command:

    python -f mem.dmp imageinfo 



In order to understand memory forensics and the steps involved. Let’s look at a case scenario, our analysis and flow will be based on the below case scenario.

Case Scenario

Your security device alerts on malicious http connection to the domain “”  which resolves to,  communication is detected from a source ip (as shown in the below screenshot).you are asked to investigate and perform memory forensics on the machine


Memory Acquisition

To start with, acquire the memory image from, using memory acquisition tools. For the sake of demo, the memory dump file is named as "infected.vmem".


Now that we have acquired "infected.vmem", let’s start our analysis using Volatility advanced memory analysis framework

Step 1: Start with what you know

We know from the security device alert that the host was making an http connection to ( So let’s look at the network connections.
Volatility’s connscan module, shows connection to the malicious ip made by process (with pid 888)


Step2: Info about

Google search shows this domain( is known to be associated with malware, probably “Rustock or TDSS Rootkit”. This indicates that source ip could be infected by any of these malwares,  we need to confirm that with further analysis.


Step 3: what is Pid 888?

Since the network connection to the ip was made by pid 888, we need to determine which process is associated with pid 888. “psscan” shows pid 888 belongs to svchost.exe.


Step4: YARA scan

Running  the YARA scan on the memory dump for the string "web3inst" confirms that this domain ( is present in the address space of svchost.exe (pid 888). This confirms that svchost.exe was making connections to the malicious domain ""


Step 5: Suspicious mutex in svchost.exe

Now we know that svchost.exe process (pid 888) was making connections to the domain "", lets focus on this process. Checking for the mutex created by svchost.exe shows a suspicious mutex "TdlStartMutex"


Step 6: Info about the mutex

Google search shows that this suspicious mutex is associated with TDSS rootkit. This indicates that the mutex "TdlStartMutex" is malicious.


Step 7: File handles of svchost.exe

Examining file handles in svchost.exe (pid 888) shows handles to two suspicious files (DLL and driver file). As you can see in the below screenshot both these files start with "TDSS"



Step 8: Detecting Hidden DLL

Volatility’s dlllist module couldn’t find the DLL starting with “TDSS” whereas ldrmodules plugin was able to find it. This confirms that the DLL (TDSSoiqh.dll) was hidden. malware hides the DLL by unlinking from the 3 PEB lists (operating sytem keeps track of the DLL's in these lists)


Step 9: Dumping the hidden DLL

In the previous step hidden DLL was detected. This hidden DLL can be dumped from the memory to disk using Volatility's dlldump module as shown below



Step 10: VirusTotal submission of dumped DLL

Submitting the dumped dll to VirusTotal confirms that it is malicious



Step 11: Looking for other malicious DLL's

Looking for the modules in all the processes that start with "TDSS" shows that msiexec.exe process (pid 1236) has reference to a temp file (which is starting with TDSS) which is suspicous.


Step 12– Suspicious DLL loaded by msiexec

Examining the DLL's loaded by the process msiexec (pid 1236) using dlllist module, shows a suspicious dll (dll.dll) loaded by msiexec process.



Step 13– Dumping DLL and VT submission

Dumping the suspicious DLL (dll.dll) and submitting to VirusTotal confirms that this is associated with TDSS (Alueron) rootkit



Step 14 – Hidden Kernel driver

In step 7 we also saw reference to a driver file (starting with "TDSS"). Searching for the driver file using Volatility’s modules plugin couldn’t find the driver that starts with "TDSS" whereas Volatility's driverscan plugin was able to find it. This confirms that the kernel driver (TDSSserv.sys) was hidden. The below screenshot also shows that the base address of the driver is "0xb838b000" and the size is "0x11000"


Step 15 - Kernel Callbacks

Examining the callbacks shows the callback (at address starting with 0xb38) set by an unknown driver



Step 16 - Examining the unknown kernel driver

The below screenshot shows that this unknown driver falls under the address range of TDSSserv.sys. This confirms that unknown driver is "TDSSserv.sys"


step 17 - Kernel api hooks

Malware hooks the Kernel API and the hook address falls under the address range of  TDSSserv.sys (as shown in the below screenshots)




Step 18 – Dumping the kernel driver

Dumping the kernel driver and submitting it to VirusTotal confirms that it is TDSS (Alureon) rootkit




Memory forensics is a powerful investigation technique and with a tool like Volatility it is possible to find advanced malware and its forensic artifacts from the memory which helps in incident response, malware analysis and reverse engineering. As you saw, starting with little information we were able to detect the advanced malware and its components.

Author bio

Monnappa K A is based out of Bangalore, India. He has an experience of 7 years in the security domain. He works with Cisco Systems as Information Security Investigator. He is also the member of a security research community SecurityXploded (SX). Besides his job routine he does reasearch on malware analysis and reverse engineering, he has presented on various topics like "Memory Forensics", "Advanced Malware Analysis", "Rootkit Analysis", "Detection and Removal of Malwares" and "Sandbox Analysis" in the Bangalore security community meetings. His article on "Malware Analysis" was also published in the Hakin9 ebook "Malware - From Basic Cleaning To Analyzing"

You can view the video demo's of all his presentations by subscribing to his youtube channel:


Hakin9 Magazine now has a course on Volatility, you should check it out ->

June 29, 2016
© HAKIN9 MEDIA SP. Z O.O. SP. K. 2013