Fundamentals
T1204Malware Analysis Fundamentals
A foundational guide to malware analysis for SOC analysts — static vs dynamic analysis, hash types (MD5/SHA1/SHA256), packers and unpacking, sandbox escape techniques, and the analyst triage workflow.
View on Graph
What Malware Analysis Is and Why Analysts Need It
- Malware analysis is the systematic examination of a suspicious file or URL to determine if it is malicious, how it operates, and what its objective is.
- MITRE ATT&CK maps user execution of malicious payloads to
T1204(User Execution), and the defense falls underM1049(Antivirus/Antimalware) andM1040(Behavioral Prevention). - Analysts who understand malware analysis can distinguish between a confirmed threat and a false positive, triage samples quickly to determine response priority, and extract indicators (IOCs) for detection and blocking.
- The discipline has two major branches: static analysis (examining the file without executing it) and dynamic analysis (running the file and observing its behavior).
Static Analysis — Examining the File Without Execution
Static analysis extracts information from the file itself without running it. It is the fastest and safest analysis method.
Hash Types — The First Check
Every file has a cryptographic hash — a fingerprint that uniquely identifies it. Analysts use three hash algorithms:
| Algorithm | Bit Length | Speed | Collision Risk | Use Case |
|---|---|---|---|---|
| MD5 | 128 bits | Fastest | Known collisions (2004) | Cached indicator lookup, de-duplication — not security-critical |
| SHA1 | 160 bits | Moderate | Theoretical collisions | Legacy compatibility, some threat intel feeds |
| SHA256 | 256 bits | Moderate | None known | Standard for all security work — use this for detection and reporting |
When to use which:
- SHA256 — Primary hash for detection rules, VirusTotal lookups, and IOC sharing (MISP, STIX)
- MD5 — Useful when searching legacy intel feeds or de-duplicating in large file collections
- SHA1 — Rarely used today; some older security tools still reference it
File Metadata Examination
Before running any tools, read the file’s metadata:
# Linux — get file type and basic info
file suspicious.exe
# PE32 executable (GUI) Intel 80386, for MS Windows
# Extract PE metadata with exiftool
exiftool suspicious.exe
# Windows — check the digital signature
sigcheck.exe -a suspicious.exe
| PE Field | What It Reveals | Suspicious Pattern |
|---|---|---|
| Compile timestamp | When the binary was compiled | Compile timestamp in the future; or 10+ years old with a recent file mod date |
| Subsystem | GUI vs Console vs Driver | GUI subsystem for a command-line tool is suspicious |
| Digital signature | Signed by a trusted CA | Missing signature, self-signed, revoked cert, or mismatched publisher name |
| Original filename | Name embedded during compilation | Mismatch between disk filename and PE metadata |
| Entry point | Starting address of executable code | Entry point in a non-standard section (not .text) suggests packing |
Section Analysis — Spotting Packers
PE executables are divided into sections. Packed malware uses non-standard sections or incorrectly sized ones:
| Section Name | Purpose | Packed Indicator |
|---|---|---|
.text | Compiled code | If .text has high entropy (>7.0), the code is likely packed |
.rdata | Read-only data (constants, imports) | Unusually large or absent suggests packing |
.data | Runtime data (global variables) | Large .data sections can hide unpacked code |
.rsrc | Resources (icons, strings, manifests) | Malware hides payloads in resources |
.UPX, .packed | Custom packer section | Direct indicator of UPX or similar packing |
.abc, .00cfg | Non-standard names | Use of arbitrary section names = likely custom packer |
Entropy check with spark or sigcheck:
# Check section entropy with sigcheck (Windows)
sigcheck.exe -e suspicious.exe
# CLI entropy scanner on Linux
cat suspicious.exe | ent
A Shannon entropy above 7.5 in the .text section strongly indicates packed or encrypted code.
Packers — Why Malware Packs Itself
Packers compress or encrypt the original executable and wrap it in a small stub that unpacks the real code in memory. Malware authors use packers to evade signature-based detection.
Common Packers
| Packer | Type | How to Detect | How to Unpack |
|---|---|---|---|
| UPX | Compressor | Section names contain .UPX | upx -d suspicious.exe |
| MPRESS | Compressor | Section .MPRESS1, .MPRESS2 | m press -d suspicious.exe |
| VMProtect | Virtualizer | Extremely high entropy, large .text | Manual unpacking required |
| Themida | Virtualizer + obfuscator | Large binary, high entropy, anti-debug | Manual unpacking required |
| ConfuserEx | .NET obfuscator | Renamed types, string encryption | de4dot for .NET unpacking |
| Custom packer | Proprietary | Non-standard sections, high entropy | Behavioral unpacking (run, memory dump) |
Unpacking Workflow
- Simple packing (UPX, MPRESS): Use the unpacker tools above
- Custom packer: Run the malware in a sandbox, let it unpack in memory, then dump the process memory
- Dump unpacked process memory:
# Use Process Hacker (Windows) to dump process memory # Or use LiME (Linux Memory Extractor) on a live Linux sandbox # Or use Volatility to dump the process: vol -f memory.dump windows.memmap.Memmap --pid 1234 --dump
Dynamic Analysis — Running Malware in a Sandbox
Dynamic analysis executes the file in a controlled environment and records everything it does. This is the most reliable way to understand malware behavior.
Sandbox Requirements
| Requirement | Why | Implementation |
|---|---|---|
| Isolation | No network path to production systems | Separate VLAN, host-only networking, or air-gapped VM |
| Network simulation | Observe C2 behavior without real risk | Fakenet-NG, INetSim, or dnsmasq with sinkhole |
| Monitoring tools | Capture every action | ProcMon, Process Hacker, Wireshark, API Monitor |
| Snapshots | Clean state between runs | VM snapshots restored after each analysis |
| Multiple OS versions | Malware checks OS version | Keep Windows 10, 11, Server 2022 images |
What to Observe
| Behavior | How to Observe | Malicious Indicator |
|---|---|---|
| Process creation | ProcMon, Process Hacker | Spawns cmd.exe, powershell.exe, rundll32.exe |
| File creation | ProcMon, Regshot | Drops files in %TEMP%, %APPDATA%, %PROGRAMDATA% |
| Registry changes | Regshot, ProcMon | Adds Run keys, service entries, COM hijacks |
| Network connections | Wireshark, Fakenet-NG | Outbound TCP to port 80/443/53, DNS queries to suspicious domains |
| Memory injection | Process Hacker, API Monitor | VirtualAllocEx + WriteProcessMemory + CreateRemoteThread |
| Persistence | Autoruns, scheduled tasks | New service, Run key, scheduled task, WMI subscription |
Sandbox Evasion Techniques
Malware authors actively detect sandbox environments and alter behavior:
| Evasion Technique | What the Malware Does | How to Counter |
|---|---|---|
| VM detection | Checks for VMware/VirtualBox drivers, MAC OUI, BIOS strings | Configure hypervisor detection evasion in VM settings |
| Sleep evasion | Calls Sleep() for 5-30 minutes before executing payload | Use API Monitor to intercept Sleep calls, skip or reduce sleep |
| Human interaction check | Waits for mouse movement, USB insertion, or open documents | Simulate human interaction via scripts |
| Domain check | Only executes if the machine is domain-joined | Join sandbox to a test domain |
| Debugger detection | Checks for IsDebuggerPresent, NtQueryInformationProcess | Use stealthy debugger (x64dbg with ScyllaHide plugin) |
| User agent check | Checks browser or tool user agent strings | Configure sandbox with real browser fingerprints |
| Anti-sandbox blacklist | Checks for known sandbox IPs, MACs, and hostnames | Randomize MAC, hostname, and IP in sandbox |
Quick evasion check — Sysmon Event ID 1 for sleep commands:
index=windows sourcetype=WinEventLog:Sysmon EventCode=1
| search CommandLine="*sleep*" OR CommandLine="*Start-Sleep*" OR CommandLine="*timeout*"
| stats count, values(CommandLine) as Commands by Image, Computer
| where count > 3
| eval alert = "Potential sandbox evasion — " . Image . " called sleep multiple times: " . Commands
| table Computer, Image, count, alert
Analyst Triage Workflow
5-Minute Checklist
| Step | Action | What You Learn |
|---|---|---|
| 1 | Hash the file (SHA256) | Unique fingerprint |
| 2 | VirusTotal lookup | Known malicious? Detection rate? |
| 3 | Check file type (file command) | PE, ELF, script, document? |
| 4 | Check digital signature | Legitimate or forged? |
| 5 | Quick strings scan | URLs, IPs, mutexes, C2 patterns |
| 6 | Check section entropy | Packed or unpacked? |
| 7 | Run in sandbox | Behavioral analysis |
Triage Decision Matrix
| Finding | Classification | Action |
|---|---|---|
| VT 0 detections + no network + no persistence | Likely benign | Document, close |
| VT 5+ detections + creates Run key + beacons out | Commodity malware | Block C2, clean host |
| VT 10+ detections + drops additional payload | Dropper | Escalate — dropper payload needs separate analysis |
| VT 0 detections + creates admin user + connects to unknown IP | Targeted/zero-day | Escalate immediately |
Tools Every Analyst Should Know
| Tool | Analysis Type | What It Does |
|---|---|---|
| VirusTotal | Reputation | Multi-engine AV scan, community comments |
| Any.Run | Dynamic | Interactive cloud sandbox |
| ProcMon (Sysinternals) | Dynamic | Real-time file, registry, process monitoring |
| Process Hacker | Dynamic | Memory inspection, process dumping, network monitoring |
| x64dbg | Static/Dynamic | Debugger for Windows binaries |
| Ghidra | Static | NSA reverse engineering framework, decompiler |
| PEStudio | Static | PE file analysis, indicators of malicious intent |
| CAPA | Static | Identifies malware capabilities from binary analysis |
| FLOSS | Static | Extracts obfuscated strings (FLARE Obfuscated String Solver) |
| Regshot | Dynamic | Before/after registry comparison |
Related
- Malware Analysis Triage — detection and response for T1204 techniques
- EDR Basics — detection and response for T1059, T1003, T1055, T1204, T1562 techniques
- Kill Chain — covers the kill chain concepts
- Ghidra — detection and response for T1204 techniques
- REMnux — detection and response for T1204 techniques
