The TLDR

• The current state of foundation AI development, dominated by a few tech companies, is closed and anti-competitive.
• Open source software development is the alternative, but foundation AI is not feasible to develop as a traditional open source software project (e.g., Linux) because it suffers from a “resource problem” where the open source contributor is also being asked to donate compute and data costs beyond any individual’s means.
• Crypto, through ownership, incentivizes resource providers to contribute to foundation open source AI projects, solving the resource problem.
• Open source AI combined with crypto leads to better AI by enabling larger models and driving more innovation. 

Introduction

In a 2024 poll by Pew Research Center, 64% of Americans said they believe social media has a negative rather than positive impact on the country, 78% said social media companies have too much power and influence in politics today, and 83% said it’s very or somewhat likely that these platforms intentionally censor political viewpoints they disagree with. Disdain towards social media platforms is one of the few issues that unites Americans.

Looking back at how the social media experiment has unfolded over the past 20 years, it seems inevitable that we would end up here. You know the story. A handful of big tech companies captured initial attention and, most importantly, user data. While there was initially hope that this data would be open, these companies quickly changed course by shutting off access after using that data to build impenetrable network effects. This basically leads us to the current state of affairs, where fewer than ten big tech social media companies exist as little fiefdoms in an oligopoly with no incentive to change because the status quo is extremely profitable. It’s closed and anti-competitive.

Examining how the AI experiment is currently playing out, I feel like I’m watching the same movie on repeat, but much more is at stake this time. A handful of big tech companies have amassed the GPUs and data to build foundation AI models and have closed off access to these models. It’s already effectively impossible for a new entrant (that doesn’t raise billions of dollars) to build a competing version because the barriers to entry are so high — pretraining a foundation model costs billions of dollars in compute capital expenditures alone, and the same social media companies that won from the last technological boon are using their control over proprietary user data to build models competitors cannot. We are full steam ahead towards recreating what we did with social media in AI: closed and anti-competitive. If we continue down this path of closed AI, a few tech companies will have unthrottled control over access to information and opportunities. 

Open Source AI and the “Resource Problem”

If we don’t want a world of closed AI, what’s our alternative? Well, the obvious answer is building foundation models as open source software projects. We have countless examples of open source projects that built foundational software we rely on every day. If Linux shows that something as fundamental as an operating system can be built open source, why should it be any different for LLMs?

Unfortunately, foundation AI models have constraints that make them different from traditional software, which significantly stunts their viability as traditional open source software projects. Specifically, foundation AI models inherently require compute and data resources beyond any individual’s means. The upshot is that, unlike traditional open source software projects that rely on people donating their time (which is already a challenging problem), open source AI also requires people to donate resources in the form of compute and data. This is open source AI’s “resource problem.”¹

To better understand the resource problem, let’s look at Meta’s LLaMa model. Meta stands out from its competitors (OpenAI, Google, etc.) in that, instead of hiding the model behind a paid API, it publicly offers LLaMa’s weights for anyone to use for free (with some restrictions). These weights represent what the model learned from Meta’s training process and are necessary to run the model. Having the weights also allows anyone to fine-tune the model or use the model’s output as input into a new model.

While it is commendable that Meta publishes LLaMa’s weights, it is not a true open source software project. Meta is, behind closed doors, using its own compute, data, and decision-making to train the model, and it unilaterally determines when to make that model available to the world. Meta does not invite community participation from independent researchers/developers because individual community members cannot afford the compute or data resources required to train or retrain the model — tens of thousands of high-memory GPUs, data centers to house them, substantial cooling infrastructure, and trillions of tokens in training data. As put by Stanford’s 2024 Artificial Intelligence Index Report, the “escalation in training expenses has effectively excluded universities, traditionally centers of AI research, from developing their own leading-edge foundation models.” To get a sense of the cost, Sam Altman mentioned the training cost for GPT-4 was $100m, and that likely does not include capital expenditures; Meta’s capital expenditures rose $2.1 billion year over year (Q2 2024 vs. Q2 2023), driven by investments in servers, data centers, and network infrastructure related to training AI models. Consequently, while community contributors to LLaMa might have the technical ability to contribute and iterate on the fundamental model architecture, they would still lack the means to do so. 

To summarize, unlike a traditional open source software project where a contributor is only asked to contribute their time, the contributor to an open source AI project is being asked to contribute their time and significant cost in the form of compute and data. It is unrealistic to rely on goodwill and volunteerism to motivate enough parties to provide these resources. They need further incentives. The success of the 176B parameter open source LLM BLOOM, which involved 1000 volunteer researchers from 70+ countries and 250+ institutions, is probably the best counterexample for the merits of goodwill and volunteerism to develop open source AI. While it is certainly an impressive accomplishment (and one I fully support), it took a year to coordinate one training run and €3M in grants from French research agencies (and that cost does not include the capital expenditures for the supercomputer used to train the model that one of those French agencies already had access to). The process of coordinating and relying on new grants to iterate on BLOOM is too cumbersome and bureaucratic to compete at the pace of big tech labs. While it has been more than two years since BLOOM was released, I am not aware of any follow-up models this collective has produced. 

To make open source AI possible, we need to incentivize resource providers to contribute their compute and data without the cost coming out of the open source contributors’ pockets.

Why Crypto Solves Foundation Open Source AI’s Resource Problem

Crypto’s breakthrough is to use ownership to make open source software projects with high resource costs possible. Crypto solves the inherent resource problem of open source AI by incentivizing speculative resource providers with potential upside in the network, rather than requiring upfront costs from open source contributors to provide those resources.

For proof of this, look no further than the original crypto project, Bitcoin. Bitcoin is an open source software project; the code that runs it is completely open, and has been from the day the project began. But the code itself is not the secret sauce; downloading and running the Bitcoin node software to create a blockchain that only exists on your local computer doesn’t accomplish much. The software only becomes useful when the amount of compute mining blocks sufficiently exceeds that of any individual contributor’s compute power. Only then can the value-add of the software be realized: to maintain a ledger controlled by no one. Like foundation open source AI, Bitcoin also represents an open source software project that requires resources beyond any individual contributor’s means. They may require this compute for different reasons — Bitcoin to make the network tamperproof and foundation AI to iterate on the model — but the wider point is that they both need resources beyond any individual contributor’s needs to function as viable open source software projects.

The magic trick that Bitcoin, or for that matter, any crypto network, uses to incentivize participants to provide resources to the open source software project is ownership in the network in the form of a token. As Jesse wrote about in his founding thesis for Variant all the way back in 2020, ownership provides an incentive for resource providers to contribute their resources to the project in exchange for potential upside in the network. This is akin to how sweat equity is used to bootstrap a fledgling corporation — by paying early employees (e.g., founders) primarily through ownership of the business, the startup can get over the bootstrapping problem by having access to labor it could not otherwise afford. Crypto extends the idea of sweat equity to resource providers instead of just those who dedicate their time. Accordingly, Variant is fixated on investing in projects that use ownership to establish network effects, such as Uniswap, Morpho, and World.

If we want to make open source AI possible, ownership via crypto is the solution to the resource problem it faces. Researchers would be free to contribute their model design ideas to the open source project because the resources required to make their ideas a reality would be provided by compute and data providers in exchange for ownership in the project, rather than requiring these researchers to pay for prohibitively high upfront costs.² Ownership could take many different forms in open source AI, but the one I’m most excited about is ownership of models themselves, like the approach Pluralis is proposing.³

Pluralis calls this approach Protocol Models, where compute providers can contribute compute resources to train specific open source models and receive ownership in the model’s future inference revenue. Because the ownership is in a specific model and the value of the ownership is based on the inference revenue, the compute provider is incentivized to choose which models are best and not spoof training (because providing useless training would decrease the expected value on future inference revenue).⁴ The question then becomes: how can ownership be enforced on Pluralis if weights are required to be sent to compute providers for training? The answer is that model parallelism is used to distribute the model shards across workers, allowing a key property of neural networks to be exploited: it is possible to make contributions to training a larger model while only having visibility into a tiny subset of the total weights, ensuring that the complete weight set remains unextractable. And because there will be many different models trained on Pluralis, trainers will have many different weight sets making recreating the model extremely difficult. This is the core concept of Protocol Models: they are trainable, can be used, but are unextractable from the protocol (without using more computational power than would be necessary to train the model from scratch). This solves a frequent concern raised by open source AI critics, that a closed AI competitor will appropriate the toils of the open project’s labor. 

Why Crypto + Open Source = Better AI

I started this piece by describing the problems with big tech control to demonstrate why closed AI is bad from a normative standpoint. But in a world in which our online experiences are tinged with fatalism, I fear this may ring hollow for most readers. So I want to conclude by giving two reasons open source AI backed by crypto will actually lead to better AI on the merits.

First, the combination of crypto and open source AI will enable us to reach the next tier of foundation models because it will coordinate more resources than closed AI can. Our current research suggests that more resources, in the form of compute and data, means better models, which is why foundation models generally keep getting bigger and bigger.⁵ Bitcoin shows us what open source software plus crypto unlocks in terms of computing power. It is the largest and most powerful computing network in the world, orders of magnitude larger than the clouds of big tech. Crypto turns siloed competition into collaborative competition. Resource providers are incentivized to contribute their resources to solve a collective problem instead of hoarding their resources to individually (and redundantly) solve that problem. Open source AI using crypto will be able to tap into the world’s collective compute and data to build model sizes far beyond what is possible in closed AI. Companies like Hyperbolic are already showing the power of tapping into collective computing resources, where anyone can rent out GPUs on their open marketplace for less money.

Second, combining crypto and open source AI will drive more innovation. This is because if we can get over the resource problem, we can return to the highly iterative and innovative open source nature of machine learning research. Prior to the recent introduction of foundation LLMs, machine learning researchers for decades openly published their models and blueprints for replicating the models. These models generally used more limited open datasets and had manageable compute requirements, meaning anyone could iterate on them. It is through this iteration that we developed advances in sequence modeling, such as RNNs, LSTMs, and attention mechanisms, that made the “transformer” model architecture underlying current foundation LLMs possible. But this all changed with the introduction of GPT-3 (which bucked a trend from GPT-2 of not being open source) and the wild success of ChatGPT. This is because OpenAI demonstrated that if you threw enough compute and data at massive models, you could build LLMs that appear to understand human language. This created the resource problem, pricing out academics and causing big tech labs to largely stop releasing their model architectures publicly to maintain their competitive advantage. The current state of relying mainly on individual labs will limit our ability to push the boundary on what constitutes state of the art. Open source AI that is made possible with crypto will mean researchers will once again be able to continue this iterative process on cutting-edge models to discover the “next transformer.”

Footnotes

1. Necessity is the mother of invention, and there are promising new models that demonstrate the ability to build foundation LLMs with significantly less resource requirements. For example, the Chinese AI development company DeepSeek recently published a report on a new model, DeepSeek-V3, that purports to outperform LLaMa at a fraction of the compute requirements (costing $6m in compute). While these are certainly promising developments, these models still suffer from a “resource problem” as the compute required is beyond any individual’s needs. As Andrej Karpathy put it when commenting on DeepSeek-V3, “Does this mean you don’t need large GPU clusters for frontier LLMs? No . . . .”

2. This piece is only focused on the “resource problem” but ownership of course also incentivizes talent — in the case of open source AI, researchers/model designers — to contribute their labor to the open source software project as well. One way projects accomplish this is by making the talent also the primary resource provider (particularly in the early days of the project) — e.g., Satoshi Nakamoto created Bitcoin and was also the first participant to mine the blockchain and earn tokens.

3. In addition to Pluralis, I’m really excited about how other crypto x AI projects are using ownership to incentivize resource providers.

Bagel’s use of ownership/monetization to incentivize parties to contribute open source model improvements to base models in privacy-preserving ways is particularly exciting. Specifically, Bagel will allow anyone to incorporate new model layers or updates without revealing proprietary details; developers can stack these layers together like Lego pieces. These open source contributors will receive a portion of future inference revenue based on their contribution.

Instead of ownership in a model, Gensyn plans to use a network token that will capture value from any transactions run on the network, across a broader variety of machine learning tasks.

4. Per the above footnote that discusses talent as a primary resource provider early on in a project, Pluralis proposes that model designers should have an exclusive period in the early stages to provide computing resources for training the model, thereby earning ownership.

5. While this is what current research suggests, it is worth noting that it is based on a centralized conception; decentralized methods (incentivized by crypto) may shatter what we think is possible by approaching the problem from a different angle. For example, Nous DisTro recently showed it could match competitive LLM training in convergence rate while massively reducing the required bandwidth during pretraining of a 15B LLM.

Thank you to Zach Langley (Axiom), Alexander Long (Pluralis), Jasper Zhang (Hyperbolic), Bidhan Roy (Bagel), Shivani (Nous), Jeff Amico (Gensyn), and Kevin J (Operator) for thoughtful feedback and inspiration.

Thank you to my Variant colleagues Jack Gorman, Jesse Walden, and Caleb Shough for valuable discussion.

Disclaimer: This post is for general information purposes only. It does not constitute investment advice or a recommendation or solicitation to buy or sell any investment and should not be used in the evaluation of the merits of making any investment decision. It should not be relied upon for accounting, legal or tax advice or investment recommendations. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by Variant. While taken from sources believed to be reliable, Variant has not independently verified such information. Variant makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. This post reflects the current opinions of the authors and is not made on behalf of Variant or its Clients and does not necessarily reflect the opinions of Variant, its General Partners, its affiliates, advisors or individuals associated with Variant. The opinions reflected herein are subject to change without being updated.