Hugging Face Model Security: What Actually Executes When You Load a Model

Running AI models locally gives you control over your environment — but model security isn't just about the weight files themselves. It's about what code the loader executes along the way. Security researchers have found reverse-shell payloads embedded in Hugging Face model packages, and insecure loading paths can grant that malicious code the same access as your Python process. In other words, what you download might not just be a model — it might come with an uninvited guest.

Model Security Starts Before the First Response

Hugging Face is a platform for sharing machine learning models, datasets, and applications. A repository is a convenient unit of distribution: it can contain weights, the configuration needed to interpret those weights, and implementation code for custom architectures. But these components are loaded differently, so they need to be evaluated separately.

Weights are the numerical values learned during training, parsed by a tensor data reader that handles shapes and values. Python object serialization (Pickle), by contrast, describes how to reconstruct objects — which may include calls to functions. When your application grants permission to run Python code from a repository, that introduces yet another execution path. Grouping all of these under the umbrella term "model" makes installation convenient, but it also makes trust boundaries easy to overlook.

Hugging Face's Pickle scanning documentation puts the core warning plainly: "Loading pickle files can be dangerous due to arbitrary code execution attacks." The key verb is load. Saving bytes to disk and handing those bytes to an unsafe deserializer are two very different operations. A malicious package doesn't need to convince a model to leak something in its output — if the surrounding software has already run its code, the process has started "accepting visitors" before inference even begins.

Pickle Security: Unpacking the Package

Pickle stores Python objects as "reconstruction instructions," which a deserializer reads and follows in sequence. An unsafe loader may parse and invoke functions during reconstruction, giving an untrusted file influence over the host process.

Think of a model package as a parcel, and the loader as the person unwrapping it. Downloading only puts the parcel on the shelf. Loading means following the assembly instructions tucked inside. If those instructions can call arbitrary functions, the behavior shifts from "reading data" to "executing the publisher's choices."

The specific loading API, library version, and settings all matter. A safety conclusion about one insecure path cannot be extended to all model loaders. Modern restricted loading modes deliberately limit what object reconstruction is allowed to do; conversely, overriding a rejection mechanism because some unfamiliar checkpoint "requires" it reopens the boundary you were relying on.

This is also why the phrase "poisoned weights" needs some clarification. It can refer either to changing model behavior by altering numerical parameters through training or editing, or to the executable packaging and repository code distributed alongside a model. The cases discussed in this article belong to the latter category — they have their own distinct entry points, permissions, and defenses.

Understanding the Pickle risk at a technical level requires knowing how its instruction set works. Python's pickle module uses a virtual machine instruction sequence called "opcodes" to describe object reconstruction. This includes the GLOBAL opcode, which can import arbitrary Python modules and call functions within them during deserialization. An attacker only needs to embed a call to something like os.system or subprocess.Popen in the Pickle byte stream to execute arbitrary system commands the moment pickle.loads() runs — before any application-layer logic checks. This is fundamentally similar to SQL injection: the data channel doubles as an instruction channel. Because of this design, Python's official documentation explicitly states "never call pickle.loads() on data from an untrusted source" — but in practice, this warning is often buried under the convenience of model loading.

Three Concrete Cases, and Their Boundaries

Goober Two: A Documented Reverse Shell

JFrog analyzed the baller423/goober2 repository. Its Pickle payload launched a reverse shell during unsafe loading. A reverse shell initiates an outbound connection and hands command access to whoever is on the other end.

The repository was subsequently removed. JFrog also noted clues suggesting the activity may have been researcher-driven, so attribution remains uncertain. The report provides evidence of the payload's behavior, but does not establish that a criminal stole named victims' cloud credentials, nor does it support inventing victim counts.

Even so, the mechanism is concerning enough on its own. Command access obtained under the loader account's identity is a meaningful foothold — and development environments often grant that account far more privilege than a single model experiment requires.

nullifAI: An Unusual Archive Format and a Late-Arriving Error

ReversingLabs described two model samples (including glockr1/ballr7), labeled nullifAI. Both contained reverse-shell payloads that detected the running platform.

One important detail is the archive format. These samples were packaged with 7-zip, while standard PyTorch loading expects a different structure. Default torch.load would reject this packaging; the dangerous path requires decompressing first, then performing unsafe Pickle deserialization. Any diagram showing "ordinary model download, then open these samples directly" would be skipping a real prerequisite.

The report also examined corrupted Pickle streams and historical scanner blind spots. Its benign file-write test showed that earlier instructions could execute before a later error halted loading. By the time the loader started complaining, the file was already there. Following ReversingLabs' disclosure, Hugging Face removed the reported samples and updated its scanner within twenty-four hours. This case describes a historical finding and its remediation — it provides grounds for investigating why the scanner failed, not an assertion that the same bypass remains unpatched today.

Model Confusion: A Missing Directory Resolves Somewhere Else

Checkmarx's Model Confusion research demonstrated an issue involving a sample from an unnamed company. A relative identifier like checkpoints/some-model can refer to either a local path or a public model namespace. If the local path is missing, resolution may land on a remote repository with the same name.

Combine that ambiguity with unsafe loading and remote code trust (trust_remote_code), and code from the selected repository could be executed. According to Checkmarx, the company's sample was corrected within a day. The report explicitly frames this as a demonstration, and states that no malicious activity was found in the sensitive naming examples it highlighted.

Two conditions are at work here: an unexpected remote package being selected, and a loading path with execution authority being authorized. Inspecting a string to see if it looks like a local folder misses the resolution step entirely. A missing directory should not be allowed to improvise and pick its own substitute.

A reverse shell is a specific remote access technique that inverts the direction of a traditional remote login. In a conventional connection, the attacker actively connects to a listening port on the victim machine. In a reverse shell, the victim machine initiates an outbound connection to an attacker-controlled server. This distinction matters in practice: most network perimeter firewall rules tightly restrict inbound connections while being relatively permissive about outbound ones (for example, allowing all outbound HTTP/HTTPS traffic). A reverse shell exploits this asymmetry to bypass inbound filtering. Once the connection is established, the attacker gains an interactive command-line session, able to execute arbitrary commands, read files, enumerate environment variables, and move laterally — all running as the user who launched the process. This is why the article lists "an available network channel" as a separate prerequisite for credential exfiltration.

How Readable Credentials Enter This Picture

Once code is running inside a process, its reach depends on the account, filesystem mounts, and other controls surrounding that process. An accessible cloud key, a token available in the environment, or a readable private key file can all become targets. Secrets stored outside that access boundary remain outside the direct read path.

Sending secrets elsewhere also requires an available network channel. Read access and outbound connectivity are two independent conditions: blocking outbound traffic constrains exfiltration; keeping secrets out of the process prevents reading them in the first place. Pure numerical weights have neither capability.

The reverse-shell reports examined here document command-access payloads. Credential harvesting and exfiltration are conditional downstream consequences of that access — not confirmed victim outcomes from these reports. The critical links in the full chain should therefore be labeled as "possible consequences."

This is where common development configurations become awkward. A container that mounts your home directory or credentials into itself exposes those mounted resources to code running inside. Namespace boundaries have value, but the keys you deliberately placed inside are still there — a container full of secrets is just a gift basket with resource limits.

The concrete credential exposure surface in development environments is worth spelling out. Cloud provider CLI tools (such as AWS CLI or gcloud) write credentials to ~/.aws/credentials or ~/.config/gcloud/ by default. In containerized workflows, developers often pass Hugging Face tokens via the HUGGING_FACE_HUB_TOKEN environment variable, or mount .cache/huggingface/, to access private model repositories. SSH private keys typically live in ~/.ssh/. All of these paths are readable by any process running under the same user — no privilege escalation required. When a model loader runs as the developer, its access boundary is nearly identical to the developer's own. Restricting model loading to a dedicated least-privilege service account, or using a container image that doesn't mount the home directory, is essentially narrowing the answer to the question: "if loading behaves unexpectedly, what can the process reach?"

Safetensors, Patched Loaders, and Useful Isolation Boundaries

Safetensors provides a tensor data format designed specifically for safe storage — it contains no Pickle-style executable object reconstruction. Adopting it addresses the serialization path. But a repository can still bundle Python code alongside tensor files, so the code loading path should be evaluated as a separate decision.

Hugging Face's custom model loading documentation warns of the risk of unintentionally executing malicious code, introduces trust_remote_code=True, and recommends pinning a specific revision. Review the exact version you intend to run, and keep that decision stable during evaluation.

PyTorch's serialization documentation notes that as of version 2.6, torch.load defaults to weights_only=True when no pickle_module is passed. The restricted deserializer narrows the attack surface for remote code execution. The documentation does retain caveats about denial-of-service and possible memory corruption, so the practical scope of this setting is worth understanding accurately.

For unfamiliar packages, it's worth separating "fetch and inspect" from "execute." Prepare a clean environment containing only the files the experiment needs; leave personal keys outside; restrict network access to what the task actually requires; and treat trust in repository code as an explicit, reviewed choice. The genuinely useful question is: if loading behaves in a way you didn't expect, what can this process reach?

Safetensors was developed and open-sourced by Hugging Face, and its safety comes from the format's own design constraints. The file header is a plain JSON structure describing tensor data types, shapes, and byte offsets; the body is a raw numerical byte stream. The format contains no callable object references whatsoever — the parser only needs to perform memory mapping and type conversion, with no semantic space for "executing the publisher's instructions." The format also enforces an upper limit on header size (100 MB by default) to prevent parsers from being exhausted by oversized headers. Importantly, Safetensors addresses the serialization path specifically. Separate Python source files in the repository — such as modeling_xxx.py — are still imported and executed directly when trust_remote_code=True. The trust decisions for these two paths need to be evaluated independently.

Conclusion: NEEDS REVIEW

Open models are worth keeping, but permission boundaries should be drawn around the loader. These documented cases demonstrate that model distribution deserves the same scrutiny we apply to other executable inputs. Free access to models has value — and so does deciding who can access your development environment.