II · THE IDEA · ARTIFICIAL INTELLIGENCE
Open Weights, Closed Frontiers
▶ Listen · narrated
A model whose weights you can inspect and whose inference you can run locally occupies a different legal and practical territory from one you query through an API. The difference shapes what you can build.
At a glance
- What is released
- Trained parameter values, tokeniser, inference code, sometimes architecture details
- What is usually withheld
- Training data composition, exact hyperparameters, intermediate checkpoints, ablation studies
- Minimum hardware
- A 7-billion-parameter model quantised to 4 bits fits in approximately 4 GB of memory
- Licence variation
- Ranges from research-only to permissive commercial use, rarely meets Open Source Definition
Imagine someone gives you a sourdough starter. You can bake bread with it, you can smell it, you can even split it and give some to a friend. What you do not have is the recipe for creating the starter from scratch, or the notes on why this particular strain of yeast was chosen, or the log of every feeding over the past year. You have the living culture, not the process that made it. An open-weight model is similar. You get the trained parameters — the finished product of a long, expensive training run — and you can use them to run the model, fine-tune it, probe its behaviour. But you do not get the training data, the sequence of decisions, the experiments that were tried and discarded. You have the artefact, not the method. That means you can deploy it, adapt it, study it, but you cannot recreate it or verify the choices that shaped it. The access is real, but it is access to the result, not the process.
An open-weight release provides the trained parameter values, typically as a set of tensor files, along with the tokeniser, inference code, and architectural specifications. For a model like LLaMA, that means you receive approximately 13 billion floating-point numbers for the 13B variant, usually in a compressed format such as 16-bit or quantised to 8-bit or 4-bit precision. You can load these into memory, run forward passes, inspect activations, and compute gradients for fine-tuning. What you do not receive is the training dataset, the data preprocessing pipeline, the exact hyperparameter schedule, the random seed, or the intermediate checkpoints. The training run is not reproducible from the released artefacts. Fine-tuning is feasible on consumer hardware using parameter-efficient methods such as LoRA, which freeze most of the weights and train only a small adapter. Full fine-tuning requires more memory, roughly twice the model size to store gradients and optimizer states. Inference, by contrast, requires only the weights themselves, and quantisation can reduce memory requirements substantially: a 7B model at 4-bit precision fits in approximately 4 GB. The licences vary. LLaMA's initial licence restricted use to non-commercial research. Gemma's licence permits commercial use but includes a use-based restriction clause and forbids using the model outputs to train a competing model. These are not open-source licences under the OSI definition, because they do not provide the preferred form for modification — which, for a model, would include the training data and the full training recipe. The term open weights is more accurate. You have the weights, you can use them, but you do not have everything you would need to recreate them or to fully audit the decisions that produced them.
Look closer
The term is open weights, not open source
The Open Source Initiative maintains a definition that requires access to the preferred form for modification — which, for a model, would include training data, data processing code, and the full training procedure. No widely distributed foundation model has met that standard. The weights are the finished artefact, analogous to a compiled binary. You can use them, fine-tune them, inspect their behaviour, but you cannot recreate the training run or understand why a particular capability emerged. The term open weights acknowledges what is actually provided.
Licences attach restrictions that vary widely
LLaMA was released under a licence permitting research use but prohibiting commercial deployment without permission. Gemma 2, by contrast, allows commercial use but prohibits certain applications and requires attribution. Some licences restrict use if your service exceeds a monthly active user threshold. Others forbid using the model to train a competing model. The weights are available, but availability is not the same as permission, and permission is not uniform.
You can run inference, but training remains expensive
Downloading weights and running them locally costs storage and memory, not compute clusters. Fine-tuning on a specific task is feasible on consumer hardware for smaller models, especially using parameter-efficient methods. Training from scratch is not. LLaMA required thousands of GPU-hours on high-end accelerators. The economic and environmental asymmetry between training and inference defines the practical frontier: you can deploy what someone else built, but building it yourself remains out of reach for most individuals and institutions.
The story
When Meta released LLaMA in February 2023, the weights arrived as a set of downloadable files and a research paper describing the architecture. You could load the model, run it, measure its behaviour, tune it on your own data. What you could not do was reproduce it. The training corpus was not published. The exact sequence of hyperparameter choices, the data cleaning steps, the decisions made when a run diverged — all of that remained internal.
This is the characteristic shape of an open-weight release. The parameters are the product of a training run, and that run consumed resources most people do not have: LLaMA used 2048 A100 GPUs over several months. The weights encode everything the model learned, but they do not encode how to learn it again, or why particular data was chosen, or what was tried and discarded.
The practical consequence is a new kind of access. If you have enough memory — and for a 7-billion-parameter model quantised to lower precision, a modern laptop suffices — you can run the model without sending your data to a third party. You control the hardware, you control the prompts, you can inspect intermediate states if the software permits it. For medical records, legal documents, proprietary code, or any context where data governance matters, this is not a minor convenience. It is a different threat model.
But the weights alone do not tell you how the model will behave on data it has not seen, or why it favours one continuation over another, or what biases the training corpus introduced. The Gemma technical report describes the architecture and some high-level training choices, but the data mixture, the exact filtering rules, the ablation studies that justified those choices — those remain unpublished. You have the model, not the process.
The result is a kind of access that enables some things and forecloses others. You can deploy locally. You can fine-tune on domain-specific data without exposing it. You can study the model's behaviour in detail, run adversarial probes, measure fairness metrics on your own benchmarks. What you cannot do is retrain from scratch with different data, or verify that the training set excluded a particular source, or audit the decisions that shaped the model before the weights were frozen. The frontier is structural, not just economic.
Why it mattered then
LLaMA's release in February 2023 shifted the terms of the conversation. Before that, the most capable models were available only through APIs controlled by the organizations that trained them. You could query GPT-3, but you could not download it, inspect it, or run it on your own hardware. LLaMA made a 65-billion-parameter model available to researchers, and although the licence restricted commercial use, the weights spread quickly beyond the intended audience. Within weeks, people were running quantised versions on consumer hardware and fine-tuning them for specific tasks. The release demonstrated that the gap between training a foundation model and deploying one was wider than the gap between deploying one and adapting it. That asymmetry mattered. It meant that the expensive part — the initial training run — could be amortized across a large community, while the cheaper part — inference and fine-tuning — could be done locally. The economic structure of the technology began to look different.
Why it matters now
The distinction between open weights and open source remains unresolved, and it has practical consequences. If you are building a system that processes sensitive data, the ability to run a model on hardware you control is not optional. If you are in a jurisdiction with strict data residency rules, an API in another country is not viable. If you are researching model behaviour, adversarial robustness, or fairness, you need access to the weights, not just the outputs. The open-weight releases from Meta, Google, and others have made that possible for models that would otherwise be out of reach. But the term open is doing a lot of work, and it is not always the same work. Some licences permit commercial use, others do not. Some allow derivative models, others forbid them. Some require attribution, others do not. The weights are available, but the legal and practical permissions vary enough that you cannot assume access means permission. And the training process remains opaque. You can study what the model does, but not always why, or what it would have done if trained differently. The frontier is no longer whether you can run a capable model locally — you can — but what you can do with it once you have it, and what you can know about how it came to be.
The surprising detail
The LLaMA weights were released under a licence intended to restrict access to approved researchers, but within days they were circulating on public torrent sites. Meta did not pursue legal action, and the weights remained widely available. The situation was unusual: the organization had released the weights deliberately, but not to everyone, and the terms of access were subverted almost immediately. The result was a model that was legally restricted but practically accessible, and the ambiguity persisted. Later releases, including Gemma, used more permissive licences from the start, acknowledging that controlling distribution of a file, once released, is not straightforward. The lesson was not that licences are meaningless, but that enforcing them for a digital artefact that can be copied freely is a different problem from enforcing them for access to a running service.
Remember this
Open weights means you can run and inspect the model, not that you can reproduce or fully understand how it was built.
Test yourself
You have the weights for a 7-billion-parameter model and want to fine-tune it on proprietary medical records. The licence permits commercial use and derivative works. Name two distinct technical constraints and one legal consideration that still apply, even though you control the weights.
Technical constraints: first, you need enough memory to load the model and the fine-tuning gradients, which even with quantisation and parameter-efficient methods may exceed consumer hardware for larger batch sizes. Second, fine-tuning requires labelled data in a format the model can learn from, and preparing that data — cleaning, annotating, formatting — is often more expensive than the compute. Legal consideration: even though the model licence permits commercial use, the medical records are subject to their own regulations, such as HIPAA in the United States or GDPR in Europe. The fact that you are processing the data locally does not exempt you from those rules. You still need consent, audit trails, and appropriate safeguards, and the model's licence does not grant those.
Go deeper
- LLaMA: Open and Efficient Foundation Language Models · arXiv · Hugo Touvron et al. · 2023-02-27
- Gemma: Open Models Based on Gemini Research and Technology · arXiv · Gemma Team et al. · 2024-03-13
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.