This piece was co-authored with Gui Bibeau
The idea of a single corporation or government controlling artificial superintelligence is becoming increasingly plausible. This concern has fueled a movement towards decentralizing AI, drawing many builders to the intersection of crypto and AI.
Bittensor is one of the projects at the forefront of this movement. In essence, Bittensor is a decentralized platform for launching specialized networks, each designed for a specific machine learning use case or digital resource provision. Currently, Bittensor hosts over 36 subnets, each delivering unique services such as text and image generation, AI model pre-training and fine-tuning, data scraping and storage, and cloud computing.
In this piece, we aim to share our experience and research related to building subnets, providing a developer with the basics needed to create a subnet.
To do this, we’ve divided the guide into three sections:
Primer on Bittensor
How to go about designing a subnet
A technical tutorial that walks through building a subnet
Primer on Bittensor
If you're already well-versed in the fundamentals of Bittensor, you might prefer to skip ahead to the subsequent sections.
As previously noted, Bittensor serves as a decentralized platform for launching specialized networks, known as subnets. These networks facilitate access to specific machine learning services and digital resources. Essentially, each subnet is dedicated to a particular task or service, operating within a clear and standardized incentive and validation framework. This framework creates a two-sided marketplace for the service provided by the subnet.
Launching a subnet on Bittensor is permissionless, allowing anyone to do so. Each subnet is managed by an owner who is responsible for designing a unique incentive mechanism, which outlines what service contributors will be incentivized to provide and how the service will be verified.
The two main participants in a subnet are miners and validators. Miners provide the desired service (e.g., running hard drives for a storage subnet or GPUs for a model inference subnet), while validators verify the integrity and quality of the miners' work in accordance with the incentive mechanism.
That was the fifty thousand foot view. Now, let’s dig into the two foundational layers that the Bittensor network is built on.
Blockchain Layer
The blockchain layer consists of two key components: Subtensor and Bittensor Core.
Subtensor
Subtensor is Bittensor’s blockchain built on the Substrate framework, which includes its native cryptocurrency, TAO. Subtensor is responsible for:
Handling consensus and block production with Proof-of-Authority, soon to be Proof-of-Stake
Extrinsics, a.k.a transactions, which include operations like TAO transfers and staking, registering subnets, etc.
Reward calculation for network participants and subnets through Yuma Consensus
Subtensor currently does not have smart contract functionality, however, this is a feature that could be added in the future.
Bittensor Core
The Bittensor Core facilitates communication between nodes in the network, providing an abstract framework for nodes to interact and exchange information. In Bittensor terminology referred to as neuron-to-neuron communication. Let’s break down its components:
Neurons – Network nodes with integrated HTTP clients for communication.
Synapses – The data object that acts as the main vehicle to exchange information.
Axon and Dendrite – Instantiating server (Axon) and client (dendrite) network elements and exchanging Synapse objects using this server-client (Axon-dendrite). The axon module uses FastAPI library to create and run API servers.
Metagraph – A data structure containing information about the current state of a subnet, similar to a service discovery layer, responsible for providing information about participants in a subnet.
In essence, Bittensor Core abstracts away the complexities of the underlying blockchain technology, making it easier for developers to launch subnets and build external applications and services on top of subnets. So for those who don’t have prior experience in blockchain or smart contract development, you’re in luck – you won’t have acquire new blockchain programming skills.
For a deeper dive into the blockchain layer, refer to Bittensor’s docs and this article by Fish. In the docs, you’ll find “Bittensor API” mentioned, and that’s really just referring to Bittensor Core.
Subnets
Subnets are what developers build and launch on the Bittensor network. They function as self-contained digital commodity markets.
Here's a high-level overview of how a subnet operates:
A subnet is defined by the incentive mechanism the owner or team designs.
Subnet miners deliver the desired service as defined by the incentive mechanism.
Subnet validators independently evaluate the services provided by the miners, also defined by the incentive mechanism.
Subnet validators rank the performance of miners on a scale from 0 to 1, a process known as setting weights. These weights are transmitted to the blockchain using Bittensor Core/API, where they are transmitted to the Yuma Consensus mechanism on-chain.
Yuma Consensus determines the consensus weights for miners, which determines how TAO is distributed to subnet miners and validators.
Yuma Consensus
Without diving too deeply into the technical details of Yuma Consensus, it's important to understand that it enables the mining and validating aspects of subnets to run off-chain. Let's illustrate this with an example.
Consider a subnet focused on distributed model inference. In this subnet, miners only share the outputs of their models in response to user queries relayed by Validators. The validators then assign a score (weight) to a miner’s outputs and send this information on-chain to the Yuma Consensus mechanism which determines the rewards they get. This off-chain processing allows Bittensor’s mining and validation systems to handle data-heavy and compute-intensive tasks that would be impractical to execute on-chain.
Because of this separation, subnets can be written in any programming language.
What’s in it for you?
The question on everyone’s mind is, “What’s in it for me?” The answer is straightforward: the subnet owner receives a share of the TAO tokens distributed to the subnet by the protocol.
Every day, 7,200 TAO is emitted and distributed to network participants. Currently, the top 64 validators on the network collectively determine how these emissions should be allocated to subnets based on the value of the services they provide. This is changing soon to a market-based mechanism (more on that here).
The distribution of tokens allocated to subnets is as follows:
Subnet Owners: 18%
Subnet Miners: 41%
Subnet Validators: 41%
So, if a subnet you launch receives 2% of network emissions, you as the subnet owner would receive 25.9 TAO per day (math: 7200*0.02*0.18), while the miner pool and validator pool on your subnet each receive 59 TAO per day (math: 7200*0.02*0.41). This setup directly incentivizes subnet owners to continuously improve the incentive mechanisms within their subnets and enhance the quality of the services they provide.
How to Design and Launch a Subnet
The first step to launching a subnet, obviously, is to come up with an idea. This is on you to figure out, but a good approach is to think about an existing inefficient service or application and consider how it can be improved through an incentive mechanism.
Step 1 – Determine What Miners will Optimize
Once you've identified the commodity or service your subnet will provide, the next step is to determine the key performance metric(s) that will be used to measure miner success and rewards. Miners should have a clear metric to optimize for.
For example, on the following subnets, miners are competing against peers on optimizing for:
Financial Market Trading (SN 8) – Achieving the highest returns on a portfolio
Model Pre-Training (SN 9) – Attaining the lowest loss on a public dataset.
Model Inference (SN 19) – Minimizing inference response time at scale
Data Storage (SN 21) – Optimizing for storage capacity, data availability, and retrieval speeds
Miners should be rewarded based on how well they optimize for the specific metrics compared to other miners on the network.
Step 2 – Design the Incentive Mechanism
The incentive mechanism should be designed around the key performance metrics identified in step 1. This ensures that miners are incentivized to exhibit desired behaviors and outcomes within the subnet.
For each metric identified, there must be an automated method to measure how well each miner optimizes that metric(s) within the incentive mechanism. Assigning importance to each metric, a weighted formula should be devised to determine how a miner is scored. For instance, in a storage subnet, retrieval speed of data might carry a higher weight than the storage capacity of a miner, incentivizing miners to prioritize speed over total data storage.
All components of the incentive mechanism’s code should be fully transparent, avoiding closed-source code or APIs, to ensure fair evaluation of each miner and prevent unfair advantages within the subnet.
For more on incentive mechanism design, check out Fish’s piece again.
Step 3 – Develop Verification Mechanism
This is where validators come in as they are responsible for running the incentive mechanism and ensuring miners are acting honestly. In a model inference subnet, for instance, miners could provide incorrect responses or use cheaper models instead of the one specified for the task. To mitigate this risk, the subnet could employ a redundant verification mechanism. This involves randomly issuing challenges to miners, resembling typical user queries but with predetermined correct answers. This ensures miners cannot exploit the system. Miners' responses to these challenges contribute to a reputation system, influencing their rewards. Miners found to be cheating are penalized accordingly.
This is just one approach. Subnet owners' primary challenge lies in designing an incentive mechanism resilient to adversarial tactics, making it exceedingly difficult for well-financed competing R&D teams to exploit the system.
Code implementation example is shown in the tutorial section.
Step 4 – Create a Base Miner
Once the incentive mechanism is established, the next step is to develop a base miner. This serves as a starting point for contributors to begin mining on your subnet. The base miner provides an initial framework that miners can modify or use to create their own custom miners.
As miners innovate and enhance their own miners, the base miner will eventually become less competitive. Over time, as a subnet owner, you can continue to improve the base miner, raising the standard for everyone. With each improvement, miners will need to advance their versions to remain competitive and earn rewards, thereby fostering continuous innovation within the subnet.
Code implementation example is shown in the tutorial section.
Step 5 – Launch on Testnet
Before launching on the mainnet, it is crucial to first deploy your subnet on the testnet. This phase allows you to refine your incentive mechanism and identify any potential exploits that miners might find. The primary objective is to ensure your incentive mechanism is robust against adversarial attacks and fine-tune it to effectively encourage the desired actions from miners. This preparation on the testnet helps in creating a more secure and efficient environment for the eventual mainnet launch.
Coding Tutorial
Now let’s get technical. Over the past few months, we’ve experimented with building our own subnet and have gained valuable insights from the experience, which we will share here. The aim of this section is to provide a step-by-step guide for developers to launch a proof of concept on the testnet.
This section will include:
An example use case for a subnet to demonstrate the tutorial
How to setup the developer environment
Implementing the subnet
A dive into the subnet code
Subnet Idea: An LLM Powered Search Engine
In this tutorial, we’ll be creating a subnet that acts as an LLM-powered search engine, similar to Perplexity.ai, where miners are incentivized to perform and summarize web searches.
We’ve discussed earlier the ideas involved in building a subnet, but now is the time to explore how to do this with code.
The two key participants in this subnet are the miners and validators. The subnet miners will be responsible for running a web server that receives requests to perform searches. They will perform the search, generate a summary, and relay the output back to the requestor.
The subnet validators have three main jobs:
Routing work to miners – Validators have exclusive access to the miners, meaning only they can send work to the miners. This work can come from validators opening an API and accepting external requests.
Evaluating the work of miners – Validators send random jobs, or challenges, to miners based on what’s outlined in the incentive mechanism. They then objectively evaluate miners on how well they perform those jobs using a reward function (a concept familiar to ML folks), scoring them on a scale from 0 to 1.
Communicating the score to the blockchain – The 0-1 scores assigned by the validators are sent to the blockchain, where Yuma Consensus translates them into rewards to miners for their services and rewards to validators for coordinating this work.
As covered in the Primer, the communication layer between the miners and validators is essentially a FastAPI Python HTTP layer. The key components of the communication layer are:
Neurons: Network nodes with integrated HTTP clients for communication.
Synapse: The data payload (the search query or the summary of the results).
Dendrite/Axon: The sending and receiving parts of the HTTP client.
Metagraph Similar to a service discovery layer, responsible for providing information about participants in the network.
With a basic understanding of how things work, the next part of this tutorial will focus on implementing a simplified version of our LLM search subnet.
Developer Environment
The development environment for Bittensor requires you to run a local mock blockchain and all the pieces of the subnet together (miner, validator and protocol definition). Note that many of these steps need to be run only once, while a few will need to be repeated each time you start your development server(s). See our included cheatsheet for a quick reference to these commands and steps.
Install the Bittensor python package system-wide.
This package is the one that we will use both to create and interact with wallets but it is also used in the code of miners and validators to synchronize with the bittensor blockchain. Here is how to install it
Apple M chips : https://docs.bittensor.com/getting-started/installation#installing-on-apple-silicon
Apple Intel chips and Ubuntu (for windows install WSL https://learn.microsoft.com/en-us/windows/wsl/install) https://docs.bittensor.com/getting-started/installation#using-a-bash-command
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/opentensor/bittensor/master/scripts/install.sh)"
We’ll need a few extra packages on which the mock blockchain depends that otherwise we would not need:
On Mac M chips:
brew install make gcc git llvm curl openssl protobuf
On Mac, Intel, or Ubuntu:
sudo apt update sudo apt install --assume-yes make build-essential git clang curl libssl-dev llvm libudev-dev protobuf-compiler
The mock blockchain uses Rust, let’s install it:
Follow these instructions to install rust https://www.rust-lang.org/tools/install. Ensure you also export cargo to oath.
Run the mock blockchain
To do this, we need to download the subtensor repo which is the actual code of the bittensor blockchain (subtensor). We’ll build it with proof of work enabled to allow us to give ourselves some mock currency and spin up our development environment.
git clone
cd subtensor
# setup rust nightly version using the provided script
./subtensor/scripts/init.sh
# build the blockchain. This will take likely a few minutes
cargo build --release --features pow-faucet
# start the mock blockchain:
BUILD_BINARY=0 ./scripts/localnet.sh https://github.com/opentensor/subtensor.git
This is not something you will have to do each time you start your development servers. A bit later in this post, we’ll share a list of commands to start your development environment quickly.
Setup wallets with test currency
Lastly, before starting to work on our subnet, we’ll need to create wallets for 3 actors of the subnet:
The owner, the person responsible for creating and registering the subnet with the main blockchain
A miner – we need at least one person to perform the search and summarization hence we’ll create 1 wallet for them
Lastly we’ll need a validator also to coordinate the work.
Some of these commands will require you to add a password, remember that password as you will need it each time you perform some actions.
Subnet owner:
btcli wallet new_coldkey --wallet.name owner
Subnet Validator:
btcli wallet new_coldkey --wallet.name validator
btcli wallet new_hotkey --wallet.name validator --wallet.hotkey default
Subnet Miner:
btcli wallet new_coldkey --wallet.name miner
btcli wallet new_hotkey --wallet.name miner --wallet.hotkey default
Mint tokens for the mock wallets:
The Owner wallet will need 1,000,000 test TAO in order to register a subnet. Each run of this command will give you around 300,000 test TAO. So you will have to run it 4 times to get over 1,000,000 TAO. Note that if you close the subtensor script and reopen it later, the state is not saved and you will have to redo these operations.
btcli wallet faucet --wallet.name owner --subtensor.chain_endpoint ws://127.0.0.1:9946
Repeat the same now for the validator. One time should be enough this time:
btcli wallet faucet --wallet.name validator --subtensor.chain_endpoint ws://127.0.0.1:9946
And lastly for the miner:
btcli wallet faucet --wallet.name miner --subtensor.chain_endpoint ws://127.0.0.1:9946
The last step is to register a subnet to the mock blockchain. Registering a subnet essentially means we are telling the main blockchain that we have a project ready to go live and establish connection with the main chain.
In the first step we will create the subnet, as the owner.
btcli subnet create --wallet.name owner --subtensor.chain_endpoint ws://127.0.0.1:9946
With the subnet created we will register a miner to it. Registration is important as subnets have limited spots available. Each time somebody opts to join a subnet as a miner or a validator they need to spend an amount of TAO. Let’s go ahead and do that:
btcli subnet register --wallet.name miner --wallet.hotkey default --subtensor.chain_endpoint ws://127.0.0.1:9946
Let’s do the same with the validator:
btcli subnet register --wallet.name validator --wallet.hotkey default --subtensor.chain_endpoint ws://127.0.0.1:9946
Closing and reopening your development environment.
The subtensor mock blockchain is meant to be a long running process and it does not store any state. This means essentially that whenever you close your terminal where this process is running, step 7 needs to be redone.
Subnet Structure and Implementation:
With the above complete, we are finally ready to start the work on the actual subnet. The OpenTensor foundation, the core development team behind Bittensor, provides a useful template with some great starter code.
Let’s clone it with git and enter the directory
git clone https://github.com/opentensor/subtensor.git
cd subtensor
The Neurons folder contains the two starting scripts that take care of running either a miner or a validator. Once you start these scripts you will have a full running subnet. It won’t yet be getting rewarded TAO but it will function as expected. Let’s do so.
Make sure you have two terminal windows as these commands are the ones to start the development servers for the miner and validator. Both need to run in parallel.
Commands
First start the miner
python neurons/miner.py --netuid 1 --subtensor.chain_endpoint ws://127.0.0.1:9946 --wallet.name miner --wallet.hotkey default --logging.debug
Then start the validator in the other window
python neurons/validator.py --netuid 1 --subtensor.chain_endpoint ws://127.0.0.1:9946 --wallet.name validator --wallet.hotkey default --logging.debug
Let’s analyze these commands. They are the same except for the wallet we use:
python neurons/validator.py tells python which of the two you are starting
--netuid 1 tells the validator/miner that it should run on subnet #1 (currently there are 32 subnets
--subtensor.chain_endpoint ws://127.0.0.1:9946 tells the validator/miner to which blockchain it is connecting. Our own test blockchain we started earlier runs with websockets at localhost 9946
--wallet.name validator will tell the bittensor sdk which library to look for on your computer. These wallets are typically stored on ~/.bittensor. These are the wallets we created earlier.
--wallet.hotkey default this simply means to use the default hotkey. Having multiple hotkeys is possible but this should be the topic of a more advanced operations guide
--logging.debug simply means we want the most logs to be outputted. This will be useful for development.
Let's dissect the key files involved in the template, breaking down their roles and how they fit together.
1. protocol.py
Protocol.py is where you will define communication between your miner and validator. The current code in the template is just an example. This is essentially the payload that will be passed back and forth between the validator and the miner. Let’s get an overview of implementing our search summarization protocol file:
```
class SearchSummarySynapse(bt.StreamingSynapse):
roles: List[str] = pydantic.Field(
...,
title="Roles",
description="Participants involved in the search and summarization process (e.g., 'user', 'searcher', 'summarizer').",
allow_mutation=False,
)
query: str = pydantic.Field(
...,
title="Query",
description="The search query provided by the user.",
allow_mutation=False,
)
search_results: List[str] = pydantic.Field(
[],
title="Search Results",
description="A list of URLs retrieved from the search engine.",
allow_mutation=False,
)
summary: str = pydantic.Field(
"",
title="Summary",
description="The generated summary of the search results."
)
Explanation:
This SearchSummarySynapse defines a more complex protocol than the simple text-prompting example.
Roles might include "user," "searcher," and "summarizer" to represent the different stages of the process.
Query holds the user's search input.
Search_results contains the URLs returned from the search engine.
Summary is where the generated summary of the search results is stored.
2. neurons/miner.py
Now that we have established a protocol of communication between our miner and validator, we’ll need to implement the forward function in the miner. This forward function is what will get called when a validator sends miners a request. It is this function’s job to perform the actual work of the miner. Let’s explore a pseudo code implementation.
class Miner(BaseMinerNeuron):
...
async def forward(self, synapse: template.protocol.SearchSummarySynapse) -> template.protocol.SearchSummarySynapse:
# ... Your code to perform the search (e.g., using a library like 'requests')
synapse.search_results = ["https://example.com/article1", "https://example.com/article2"] # (Replace with actual search results)
# ... Your code to generate the summary (e.g., using a language model)
synapse.summary = "This search explored the latest advancements in AI..." # (Replace with your model's output)
return synapse
Explanation:
The forward function receives as an argument the synapse which is an instance of our protocol.
It performs a search, for example we could call an API that performs a web search or even implement a web scraping technique. Typically as a subnet owner you will provide a base implementation but miners are free to optimize their code and techniques for more efficient results.
At the end, you return the synapse having filled the summary property. This will get sent over to the validator for evaluation.
3. neurons/validator.py
The last file we truly care about is the validator file and especially its forward function. This function will have to perform the following action:
Get a list of miners to send work to perform.
Prepare a synapse which will describe the work. For example sending our search query.
Await the results to evaluate them.
Communicate with the blockchain the evaluations to reward miners.
class Validator(BaseValidatorNeuron):
...
async def forward(self):
miner_uids = get_random_uids(self, k=self.config.neuron.sample_size) # Select miners
responses = await self.dendrite(
axons=[self.metagraph.axons[uid] for uid in miner_uids],
synapse=SearchSummarySynapse(roles=["user", "searcher", "summarizer"], query="What is the meaning of life?"),
deserialize=True,
)
# ... Code to score responses and update weights
rewards = get_rewards(self, query=self.step, responses=responses)
self.update_scores(rewards, miner_uids)
The important aspects to understand from these three files is that a subnet is mostly a closed loop of work. Validators generate a bill of work, miners execute that work and we close the loop by having validators evaluate it. At the end of the cycle, this process is repeated.
The key to creating a successful subnet is that the work should lead to a healthy competition between the different miners. Your miners should strive to improve and iterate on your original implementation. The best miner gets rewarded with the most TAO, so all miners strive to be the best.
Alright, let's craft a pseudo-code implementation for the get_rewards function that evaluates the search summaries produced by miners and assigns a score from 0 to 1.
def get_rewards(
self,
query: str,
responses: List[SearchSummarySynapse],
) -> torch.FloatTensor:
Calculates a reward score for each miner based on the quality of their search summaries.
Args:
query (str): The original search query sent to the miners.
responses (List[SearchSummarySynapse]): A list of responses from the miners.
Returns:
torch.FloatTensor: A tensor of reward scores (between 0 and 1) for each miner.
"""
rewards = []
for response in responses:
score = 0.0 # Initialize score
# 1. Relevance Evaluation
# - Compare the summary to the original query.
# - Use a similarity metric (e.g., cosine similarity, Jaccard similarity) to assess how relevant the summary is to the query.
# - Normalize the similarity score to a value between 0 and 1.
relevance_score = calculate_relevance_score(response.summary, query)
score += relevance_score * 0.5 # Weigh relevance heavily
# 2. Coherence Evaluation
# - Assess the coherence and readability of the summary.
# - Use a coherence metric (e.g., ROUGE, BLEU, BERTScore) to check if the summary is well-structured and easy to understand.
# - Normalize the coherence score to a value between 0 and 1.
coherence_score = calculate_coherence_score(response.summary)
score += coherence_score * 0.3 # Weigh coherence
# 3. Informativeness Evaluation
# - Check if the summary captures the key information from the search results.
# - Compare the summary to the search results (e.g., using a sentence embedding model, compare the vectors of each sentence in the summary to the search results).
# - Normalize the informativeness score to a value between 0 and 1.
informativeness_score = calculate_informativeness_score(response.summary, response.search_results)
score += informativeness_score * 0.2 # Weigh informativeness
# 4. Factual Accuracy (Optional)
# - If applicable, assess the factuality of the summary.
# - Use a fact-checking tool or knowledge base to verify the claims in the summary.
# - Adjust the score based on factual accuracy.
factual_accuracy_score = calculate_factual_accuracy_score(response.summary) # (Optional)
score += factual_accuracy_score * 0.1 # Weigh factual accuracy (if used)
rewards.append(score)
return torch.FloatTensor(rewards).to(self.device)
Explanation:
Initialization: The code initializes a list called rewards to store the scores for each miner.
Iterating Through Responses: The code loops through the responses (a list of SearchSummarySynapse objects) from the miners.
Calculating Scores: For each response, the code calculates a score based on four factors: relevance, coherence, informativeness, and optionally, factual accuracy. Each factor contributes to the overall score, and the weights (0.5, 0.3, 0.2, 0.1) reflect the importance of each factor in determining the quality of the summary.
Adding to Rewards: The calculated score for each miner's summary is appended to the rewards list.
Returning Rewards: The function returns the rewards list as a PyTorch tensor, ready to be used by the validator for updating scores and distributing TAO.
Important Considerations:
Metric Selection: You'll need to choose appropriate metrics for each evaluation factor. The choices will depend on the nature of your search and summarization tasks. For example, for factual accuracy, you might use a knowledge base or a fact-checking API.
Weighting: Experiment with different weightings for the evaluation factors to reflect the priorities of your subnet.
Normalization: Make sure to normalize the scores for each factor to a range between 0 and 1 to ensure consistent scaling.
Next Steps to Consider
After developing a first version of your subnet locally, the next step is to launch the subnet on the Bittensor Testnet.
Register on Testnet
So far, everything above has been done on a local version of the blockchain. When you are feeling confident about the basics working locally, you should register the subnet on testnet. This testnet is a completely free test environment where you can get test TAO for free and safely experiment and test drive your project.
On the testnet, other miners can begin experimenting with the subnet and expose any potential exploits for you to fix. Finding miners to do so can be done via the Bittensor discord server.
Create an API Layer on the Validators
An important step many subnets take is to open access to the subnet’s services through an API to create monetization opportunities. Good options for this would be fastAPI or other API layers in python. This is something that normally happens when you want to create a product on top of your subnet.
Once created, the Request Network by Taoshi can be used by validators to advertise their bandwidth to the subnet and earn money from external users using their API.
Final Thoughts
At OSS Capital, we believe Bittensor will play a pivotal role in decentralizing AI and digital commodity markets. If you’re considering launching a subnet or have any questions, feel free to email me at sami@oss.capital, or DM me on twitter.