Cyber For AI Notes

Lab: Malicious Model Identification (Pickle Analysis)

Exercise to showcase how model loading can be compromised by unsafe deserialisation

June 20, 2026

Exercise: mountains — Malicious Model Identification (Pickle Analysis)

MITRE ATLAS: AML.T0051 (Model Poisoning), AML.T0049 (Unsafe Deserialization), AML.T0053

Objective

Analyze the four ML model files and identify:

  • Which models are malicious
  • What each malicious model does
  • Why the behavior is dangerous

The exercise focuses on static analysis only — no execution of untrusted models.

Scenario

You receive 4 serialized model files (e.g. .pkl, .pt, .joblib) from unknown third-party sources.
They claim to be trained ML models.

You must determine which are safe to load and which are malicious.

Constraints

  • Do not load the models normally
  • Do not execute any untrusted code
  • Static inspection only

Skills Learned

  • Using pickletools for static analysis
  • Reading and interpreting REDUCE opcodes
  • Identifying deserialization attack patterns
  • Distinguishing safe vs unsafe loading paths

Challenge

Step 1: Identify Serialization Format

For each file:

  • Determine whether it is a Pickle-based format
  • Identify framework hints (torch, sklearn, joblib, custom objects)

Commands you may use:

file model_X
strings model_X | head

Step 2: Static Pickle Inspection

Disassemble each model using:

python -m pickletools model_X.pkl

Look specifically for:

  • GLOBAL
  • REDUCE
  • STACK_GLOBAL
  • BUILD
  • INST

Step 3: Analyze REDUCE Opcodes

For each REDUCE opcode:

  • Identify the function or callable being invoked
  • Trace its arguments
  • Determine whether it can cause side effects

Red flags include:

  • os.system
  • subprocess.Popen
  • eval, exec
  • builtins.open
  • Network-related libraries
  • Shell commands

Step 4: Classify Each Model

For each of the 4 models, answer:

  • Is it malicious? (Yes / No)
  • What happens during deserialization?
  • What capability does the attacker gain?
  • At what point does execution occur?

Step 5: Compare Safe vs Unsafe Loading

Evaluate:

  • Why pickle.load() is dangerous
  • Why torch.load() inherits the same risk
  • Which loading methods mitigate the issue (e.g. weights_only=True, safetensors)

Deliverables

Produce a short report containing:

  • A table summarizing all 4 models
  • Opcode evidence for malicious behavior
  • Clear explanation of the attack mechanism

Example table format:

| Model | Malicious | Evidence | Effect | |:-----:|:---------:|:-------------------:|:------------------------:| | A | Yes | REDUCE →os.system | Remote command execution | | B | No | Pure tensor storage | Safe | | C | Yes | REDUCE → eval | Arbitrary code execution | | D | No | No GLOBAL/REDUCE | Safe |

Key Insight

Pickle is code execution, not a data format.

Any model that triggers REDUCE with attacker-controlled callables can execute arbitrary code during loading, before inference, before validation, and without explicit user action.

Safe ML pipelines treat model files as untrusted binaries, not data.