II · THE IDEA · ARTIFICIAL INTELLIGENCE
Model Formats: GGUF, Safetensors, MLX
▶ Listen · narrated
Download a model and you receive a file several gigabytes large. Inside are the weights—but also metadata, alignment choices, and decisions about precision that were made before you ever ran an inference.
At a glance
- What they contain
- Tensor arrays (the weights), metadata, and format-specific information about precision and layout
- Safetensors
- Created by Hugging Face to replace pickle; header-only metadata, no code execution
- GGUF
- GGML's format; stores quantisation method and parameters directly in the file
- MLX
- Apple's format optimised for unified memory on Apple Silicon
Think of a model file as a very large spreadsheet saved to disk. Each row is a tensor—a named array of numbers—and the file format is the rules for how those rows are written and read back. An old format called pickle could save not just numbers but instructions, which meant opening a file could run hidden code. That was dangerous, so a new format called safetensors was created that saves only numbers and labels, nothing that can execute. Another format, GGUF, does something extra: it saves the numbers in a compressed form and includes a recipe for decompressing them. This makes the file much smaller, but you need the recipe to use it. The format you choose determines whether the file is safe, how fast it loads, and whether it fits in your computer's memory.
A model file serialises the model's state dictionary: a mapping from parameter names to tensor values. Safetensors stores this as an eight-byte length-prefixed JSON header followed by raw tensor data in C-contiguous layout, using standard dtypes (float32, float16, bfloat16, int8, etc.). The header includes each tensor's name, shape, dtype, and byte range, enabling zero-copy memory-mapped loading. No custom deserialisers or class constructors are invoked. GGUF extends this by storing quantisation metadata in a key-value header: which tensors use which quantisation scheme (Q4_0, Q5_K_M, etc.), block sizes, scaling factors, and bit-packing formats. The loader dequantises on access, either in memory or lazily. GGUF also supports alignment padding to place tensors at SIMD-friendly offsets. MLX uses a similar header-plus-data structure but optimises layout for unified memory: tensors are stored in a format that both CPU and GPU can read without reformatting, and the framework's lazy evaluation means tensors are not materialised until a computation forces it. All three formats support memory mapping, but only safetensors guarantees that the on-disk layout is the in-memory layout for all supported dtypes. GGUF and MLX may require transformation at load time. Converting between formats is lossy if quantisation is involved: you can dequantise GGUF to safetensors, but you lose the quantisation metadata and the file size advantage. Converting safetensors to GGUF requires re-quantising, which is a separate, sometimes lengthy, process.
Look closer
Safetensors was a security response
The older pickle format, standard in Python, can execute arbitrary code during deserialisation. That meant downloading a model file was downloading executable instructions, not just data. An attacker could embed malicious code in a weight file and it would run the moment you loaded the model. Safetensors prevents this by design: the format contains only a JSON header and raw tensor bytes, with no mechanism for code execution. The header describes the tensors—name, shape, data type, byte offset—but never calls a function.
GGUF encodes quantisation in the file itself
When you quantise a model to four bits per weight, that choice must be recorded somewhere. GGUF stores the quantisation method as metadata in the file header: which layers use which precision, what the scaling factors are, how the bits are packed. This means a GGUF file is not just weights but a complete specification of how to reconstruct the inference-time numbers. A safetensors file, by contrast, stores only the tensors as they are; quantisation happens separately, either before saving or during a conversion step.
Format determines memory layout and loading speed
Safetensors uses zero-copy deserialisation: the file is memory-mapped, and tensors are read directly from disk without an intermediate buffer. This makes loading nearly instantaneous for large models. GGUF supports memory mapping too, but also includes alignment and padding rules so that tensors sit at addresses convenient for SIMD operations. MLX, designed for Apple's unified memory architecture, stores tensors in a layout optimised for the GPU and CPU sharing the same physical RAM. The format is not just a container; it is an optimisation decision frozen at save time.
The story
A model file is a serialised snapshot of millions or billions of floating-point numbers, organised into tensors—multi-dimensional arrays with names like `model.layers.0.self_attn.q_proj.weight`. The format determines how those numbers are written to disk, what metadata accompanies them, and what guarantees you have about safety and compatibility when you load them back.
For years, PyTorch models were saved using Python's pickle protocol. Pickle is general-purpose: it can serialise almost any Python object, including functions and classes. That flexibility became a liability. Because pickle can reconstruct arbitrary objects, a malicious actor could craft a weight file that executed code during loading—deleting files, exfiltrating data, or installing malware. The risk was not hypothetical: security researchers demonstrated exploits, and the community recognised that treating model files as pure data was a mistake if the format allowed them to behave as programs.
Hugging Face introduced Safetensors in 2022 as a deliberate constraint. The format begins with an eight-byte little-endian integer specifying the header length, followed by a JSON header containing only metadata—tensor names, shapes, data types, and byte offsets into the file. After the header comes raw binary data: the tensor values themselves, laid out contiguously with no intervening structure. There is no provision for custom deserialisation logic, no hook for running initialisation code. You get exactly what the bytes encode, nothing more.
The result is both safer and faster. Because the header declares where each tensor lives in the file, a loader can memory-map the entire file and construct tensor views without copying data into a separate buffer. For a fifty-gigabyte model, this turns a multi-minute load into a sub-second operation. The trade-off is simplicity: safetensors cannot store Python objects, training state, or optimizer parameters in the same file. It is a weight format, not a checkpoint format.
GGUF, the format used by llama.cpp and the broader GGML ecosystem, makes different choices. It was designed for CPU inference and for models that have been quantised to low precision—four bits, five bits, sometimes lower. The file begins with a magic number and version field, then a key-value metadata section that can store arbitrary information: the model's architecture, its tokeniser configuration, the quantisation scheme applied to each layer, alignment requirements, even generation parameters like temperature defaults.
That metadata is weight-bearing. A GGUF file does not just contain tensors; it contains instructions for reconstructing the inference-time representation of those tensors. If a layer's weights have been quantised to four bits using a particular blocking scheme, the file records the block size, the scaling factors, and the bit-packing order. The loader reads this metadata and knows how to expand the compressed representation back into the numbers the model expects. This makes GGUF files self-contained in a way safetensors files are not: you can share a single GGUF file and the recipient has everything needed to run inference, including the quantisation recipe.
MLX, Apple's machine learning framework, introduced its own format optimised for unified memory. On Apple Silicon, the CPU and GPU share physical RAM, and tensors do not need to be copied between devices. The MLX format stores tensors in a layout that both processors can read efficiently, and the framework's lazy evaluation model means tensors are not materialised until they are actually needed. The format is less widely adopted outside the Apple ecosystem, but it reflects the same principle: the file format is not neutral. It encodes assumptions about the hardware, the precision, and the inference strategy.
All three formats support memory mapping, but the guarantees differ. Safetensors guarantees that the file layout matches the in-memory layout for standard tensor types, so mapping is always safe. GGUF files may require decompression or dequantisation, in which case memory mapping provides the compressed data and the loader expands it on access. MLX files assume unified memory and may not be directly usable on systems with discrete GPUs.
The choice of format also determines ecosystem compatibility. Safetensors is the standard for Hugging Face's model hub and is supported by most training frameworks. GGUF is the standard for llama.cpp and its many descendants, and for anyone running models on consumer hardware without a GPU. MLX is specific to Apple's framework. Converting between formats is possible but not always lossless: converting a quantised GGUF file to safetensors requires either dequantising back to full precision or storing the quantised weights as opaque blobs, losing the self-describing metadata that made the GGUF file portable.
The format is not an afterthought. It is the interface between the model as trained and the model as deployed, and it determines what optimisations are possible, what risks you accept, and what hardware you can target.
Why it mattered then
The shift from pickle to safetensors was a response to a specific threat model: models were being distributed as untrusted artefacts, downloaded from repositories by users who had no way to audit them. The pickle format made every model file a potential vector for remote code execution, and the scale of the problem grew as open-weight models became common. Hugging Face, as the largest host of model files, had both the incentive and the reach to introduce a safer standard. Safetensors was not a novel idea—storing tensors as raw bytes with a metadata header is straightforward—but it required coordination. The format succeeded because Hugging Face provided conversion tools, made safetensors the default for new uploads, and convinced framework maintainers to add native support. Within a year, most new models were distributed in safetensors format, and the security risk receded. The decision to create a new format rather than patch pickle reflected a judgement that pickle's flexibility was the problem, not a feature to be preserved.
Why it matters now
Format choice still determines what you can do with a model. If you are running inference on a laptop with sixteen gigabytes of RAM, you need a quantised model, and GGUF is the most common format for that use case. If you are fine-tuning a model on a cloud GPU, you need safetensors because that is what the training frameworks expect. If you are deploying on Apple Silicon, MLX may give you better performance than either alternative. The format also affects trust: a safetensors file is easier to audit because it cannot execute code, while a GGUF file requires trusting the metadata it declares. As models grow larger and quantisation techniques become more sophisticated, the format becomes the place where those techniques are encoded. A model is not just its weights; it is its weights plus the recipe for using them, and the format is where that recipe lives.
The surprising detail
GGUF's predecessor, GGML's original format, used a different magic number and metadata structure. When the format was revised, the version number was incremented and the magic number changed, but older llama.cpp versions cannot read newer GGUF files even if the model architecture is identical. This is not a bug but a deliberate compatibility break: the format change allowed new quantisation schemes that older loaders would misinterpret, producing incorrect outputs rather than failing cleanly. The decision to break compatibility reflects a judgement that silent wrongness is worse than loud failure. It also means that GGUF files are versioned artefacts, and a file that works today may require a loader update tomorrow.
Remember this
The format is not packaging. It encodes precision, layout, and safety decisions that determine what you can do with the model and what risks you accept.
Test yourself
You download two files of the same model: one safetensors at 28 GB, one GGUF at 4.1 GB. Both load successfully. What explains the size difference, and what trade-off has been made?
The GGUF file is quantised—the weights have been compressed from their original precision, likely sixteen-bit floats down to four or five bits per weight. The safetensors file stores the weights at their trained precision. The trade-off is memory and speed against accuracy: the quantised model uses less RAM, loads faster, and runs faster, but produces slightly different outputs because the weights are approximations. The GGUF file also encodes the quantisation scheme in its metadata, so the loader knows how to reconstruct the inference-time values. The safetensors file is closer to the trained model, but may not fit in memory on consumer hardware.
Go deeper
- ggml/docs/gguf.md at master · ggml-org/ggml · GitHub · github.com
- Safetensors · Hugging Face · huggingface.co
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.