Reshaping the Boundaries of Computing: The Current Situation and Prospects of Decentralized Computing Power

来源：Future3 Campus

With the development of AI and other fields, many industries will achieve great changes from the underlying logic, computing power will rise to a more important position, and all aspects related to it will also cause extensive exploration in the industry.

Computing power required

Since the release of Avatar in 2009, Weta Digital has contributed to the visual effects rendering of the entire film with unparalleled realism. On its 10,000-square-foot server farm in New Zealand, its computer fleet processes up to 1.4 million tasks per day and 8 gigabytes of data per second, and even then it runs continuously for more than a month before it completes all the rendering work.

With large-scale machine calls and cost investment, “Avatar” has achieved outstanding feats in the history of cinema.

On January 3 of the same year, Satoshi Nakamoto mined the genesis block of Bitcoin on a small server in Helsinki, Finland, and received a block reward of 50 BTC. Since the first days of cryptocurrency, computing power has played a very important role in the industry.

The longest chain not only serves as proof of the sequence of events witnessed, but proof that it came from the largest pool of CPU power.

—— Bitcoin Whitepaper

In the context of the PoW consensus mechanism, the contribution of computing power provides a guarantee for the security of the chain. At the same time, the continuously rising hashrate can also support the continuous investment in computing power and positive revenue expectations of miners. The real demand for computing power in the industry has also greatly promoted the development of chip manufacturers. The mining machine chip has gone through the development stages of CPU, GPU, FPGA, ASIC and so on. At present, Bitcoin mining rigs are usually chips based on ASIC (Application Specific Ingrated Circuit) technology that can efficiently execute specific algorithms, such as SHA-256. The huge economic benefits brought by Bitcoin have also driven the demand for computing power for related mining to rise, but the overly specialized equipment and cluster effect have caused the siphon effect of its own participants, whether it is miners or mining machine manufacturers, showing a capital-intensive concentration development trend.

With the advent of Ethereum’s smart contracts, with its programmability, composability and other characteristics, it has formed a wide range of applications, especially in the field of DeFi, making the price of ETH bullish all the way, and the mining difficulty of Ethereum, which is still in the PoW consensus stage, has also been rising. Miners also have increasing computing power requirements for Ethereum mining machines, but unlike Bitcoin, which uses ASIC chips, Ethereum needs to use graphics processing units (GPUs) for mining calculations, such as the Nvidia RTX series. In this way, it is more suitable for general-purpose computing hardware to participate, which even triggered the market competition for GPUs, which led to the situation that high-end graphics cards on the market were out of stock for a while.

When the time came to November 30, 2022, ChatGPT developed by OpenAI also demonstrated the epoch-making significance in the field of AI. In the new version launched in September this year, generative AI with multimodal features such as speech and images has brought the user experience to a newer stage.

However, GPT4 has more than one trillion parameters involved in model pre-training and subsequent fine-tuning. These are the two parts of the AI space that require the most computing power. In the pre-training phase, learn a large amount of text to master language patterns, grammar, and contextual context. Enables them to understand linguistic patterns to generate coherent and contextual text based on input. After pre-training, GPT4 can be fine-tuned to better adapt to specific types of content or styles, improving performance and specialization in specific demand scenarios.

Due to the Transformer architecture adopted by GPT, the self-attention mechanism is introduced, which enables the model to pay attention to the relationship between different parts of the sequence while processing the input sequence, so the demand for computing power has increased dramatically, especially in the processing of long sequences, which requires a large number of parallel computing and storage of a large number of attention scores, so it also requires a large amount of memory and high-speed data transmission capacity. At present, mainstream LLMs of the same architecture have a huge demand for high-performance GPUs, which also indicates that the investment cost in the field of large AI models is huge. According to the speculation of related SemiAnalysis, it is estimated that the cost of GPT4 model training is as high as 63 million US dollars. In order to achieve a good interactive experience, GPT4 also needs to invest a lot of computing power in its daily operation to maintain its daily operation.

Computing Hardware Classification

Here we need to understand the main types of computing hardware, and what kind of computing power demand scenarios can be handled by CPU, GPU, FPGA, and ASIC.

• From the schematic diagram of CPU and GPU architecture, GPU contains more cores, which enables GPU to process multiple computing tasks at the same time, and parallel computing has stronger processing power, which is suitable for processing a large number of computing tasks, so it has been widely used in the field of machine learning and deep learning. On the other hand, CPUs have a smaller number of cores and are suitable for processing a single complex computation or sequential task more intensively, but are not as efficient as GPUs when dealing with parallel computing tasks. In rendering tasks and neural network computing tasks, a large number of repetitive and parallel computations are often processed, so GPUs are more efficient and applicable than CPUs in this regard.

• Field Programmable Gate Array (FPGA) is a semi-custom circuit in the field of application-specific integrated circuits (ASICs). Consisting of an array of a large number of small processing units, FPGAs can be understood as programmable digital logic circuit integration chips. The current application is mainly focused on hardware acceleration, and other tasks are still done on the CPU, allowing the FPGA and CPU to work together.

• ASIC (Application Specific Integrated Circuit) refers to an integrated circuit designed to meet the requirements of specific users and the needs of specific electronic systems. Compared with general-purpose integrated circuits, ASICs have the advantages of smaller size, lower power consumption, improved reliability, improved performance, enhanced confidentiality, and reduced cost when mass-produced. Therefore, in the inherent scenario of Bitcoin mining, only specific computing tasks need to be performed, and ASIC is the most suitable. Google has also launched a Tensor Processing Unit (TPU) specifically designed for machine learning as a type of ASIC, but currently mainly provides computing power rental services through Google Cloud.

• ASICs are application-specific integrated circuits compared to FPGAs, and the integrated circuits are fixed once the design is complete. FPGAs, on the other hand, integrate a large number of digital circuit basic gates and memories within an array, and developers can define circuits by flashing the FPGA configuration, and this flashing is replaceable. However, with the current update speed in the AI field, customized or semi-customized chips cannot be adjusted and reconfigured in time to perform different tasks or adapt to new algorithms. As a result, the universal adaptability and flexibility of GPUs make them shine in the field of AI. Major GPU vendors have also made relevant optimizations for the adaptation of GPUs in the AI field. Nvidia, for example, has introduced the Tesla series of GPUs and Ampere architecture GPUs designed for deep learning, which contain hardware units (Tensor Cores) optimized for machine learning and deep learning computing, which enable GPUs to perform forward and backpropagation of neural networks with greater efficiency and lower power consumption. In addition, a wide range of tools and libraries are provided to support AI development, such as CUDA (Compute Unified Device Architecture) to help developers leverage GPUs for general-purpose parallel computing.

Decentralized Computing Power

Decentralized computing power refers to the way in which processing power is provided through distributed computing resources. This decentralized approach is often combined with blockchain technology or similar distributed ledger technology to pool and distribute idle computing resources to users who need them for resource sharing, transactions, and management.

Generate Background

• Strong demand for computing hardware. **The prosperity of the creator economy has made the direction of digital media processing enter the era of national creation, and the surging demand for visual effects rendering has appeared in the form of specialized rendering outsourcing studios and cloud rendering platforms, but this method also requires itself to invest a lot of money in the early stage of computing hardware procurement.

• **Single source of computing hardware. **The development of the AI field has intensified the demand for computing hardware, and GPU manufacturers around the world, led by Nvidia, have made a lot of money in this AI computing power race. Its supply capacity has even become a key factor that can hinder the development of a certain industry, and Nvidia’s market capitalization exceeded $1 trillion for the first time this year.

• **The provision of computing power still mainly relies on centralized cloud platforms. **At present, the centralized cloud vendors represented by AWS are really benefiting from the surge in demand for high-performance computing, which have launched GPU cloud computing services, taking the current AWS p4d.24xlarge as an example, renting an HPC server specializing in ML, including 8 Nvidia A100 40GB GPUs, which costs $32.8 per hour, and its gross profit margin is estimated to reach 61%. This has led other cloud giants to jump in and hoard hardware to take advantage of the industry’s early development.

• **Political, human intervention and other factors lead to unbalanced development of the industry. **Unbalanced: It is not difficult to see that the ownership and concentration of GPUs is more skewed towards organizations and countries with abundant funds and technology, and is dependent on high-performance computing clusters. As a result, chip semiconductor manufacturing powerhouses represented by the United States are also imposing stricter restrictions on the export of AI chips to weaken the research capabilities of other countries in the field of general artificial intelligence.

• The allocation of computing resources is too centralized. ** The development initiative in the field of AI is in the hands of a few giant companies, and the giants represented by OpenAI, with the blessing of Microsoft, are backed by the rich computing resources provided by Microsoft Azure, which makes OpenAI’s every new product release a reshaping and integration of the current AI industry, making it difficult for the rest of the team to match it in the field of large models.

So in the face of high hardware costs, geographical restrictions, and uneven industrial development, are there other solutions?

**Decentralized computing platforms were created with the aim of creating an open, transparent, and self-regulating marketplace for more efficient use of global computing resources. **

Adaptive Analysis

1. Decentralized computing power supply side

At present, the high price of hardware and the artificial control of the supply side have provided soil for the construction of decentralized computing power networks.

• From the perspective of the composition of decentralized computing power, a variety of computing power providers range from personal PCs, small IoT devices to data centers, IDCs, etc., and a large amount of accumulated computing power can provide more flexible and scalable computing solutions, thereby helping more AI developers and organizations to make more efficient use of limited resources. Decentralized computing power sharing can be realized through the idle computing power of individuals or organizations, but the availability and stability of these computing power are limited by the use restrictions or sharing ceilings of their own users.

• **The potential source of high-quality computing power is the computing power resources directly provided by the transformation of relevant mining farms after Ethereum is converted to PoS. Coreweave, a leading GPU-integrated computing power provider in the United States, is the largest mining farm for Ethereum in North America, based on a well-established infrastructure. In addition, the retired Ethereum mining machines, which also contain a large number of idle GPUs, it is reported that there are about 27 million GPUs working on the network at the peak of the Ethereum mining era, and revitalizing these GPUs can further become an important source of computing power for the decentralized computing network.

2. Demand side of decentralized computing power

• From the perspective of technical implementation, decentralized computing resources in graphics rendering, video transcoding, this kind of computational complexity of the task, combined with blockchain technology and web3 economic system can ensure the safe transmission of information and data, for the network participants to bring tangible revenue incentives, accumulated effective business models and customer groups. The AI field involves a large number of parallel computing, communication and synchronization between nodes, etc., and has very high requirements for the network environment, so the current applications are also concentrated in fine-tuning, inference, AIGC and other more application layers.

• From the perspective of business logic, the market for pure computing power trading is unimaginative, and the industry can only roll up the supply chain and pricing strategies, but these happen to be the advantages of centralized cloud services. For example, Render Network and 2023 Q1 have also launched a natively integrated Stability AI toolset, which allows users to introduce Stable Diffusion jobs with this feature, and their business is no longer limited to rendering jobs but expands to the AI field.

• From the perspective of the main customer groups, it is obvious that large B-end customers will be more inclined to centralized integrated cloud services, they usually have sufficient budgets, they are usually engaged in the development of underlying large models, and need more efficient forms of computing power aggregation; therefore, decentralized computing power is more for small and medium-sized development teams or individuals, engaged in model fine-tuning, or application layer development, and there are no too high requirements for the form of computing power provided. They are more price-sensitive, decentralized computing power can fundamentally reduce the initial cost of investment, so the overall cost of use is also lower, based on Gensyn’s previous estimated cost, converting computing power into the equivalent computing power provided by V100, the price of Gensyn is only $0.4 per hour, compared with the same type of computing power required by AWS $2 per hour, which can be reduced by 80%. Although this part of the business does not account for the majority of the current industry spending, as the use scenarios of AI applications continue to expand, the future market size should not be underestimated.

• From the perspective of the services provided, it can be found that the current project is more like the concept of a decentralized cloud platform, providing a complete set of management from the whole process of development, deployment, launch, distribution and transaction, which has the advantage of attracting developers, using relevant tool components to simplify development and deployment, improve efficiency, and at the same time attract users to use these complete application products on the platform, forming an ecological moat based on its own computing power network. However, at the same time, it also puts forward higher requirements for project operation. How to attract and retain excellent developers and users is particularly important.

Applications in different fields

1. Digital Media Processing

Render Network A blockchain-based global rendering platform whose goal is to empower creators with digital creativity. It allows creators to scale GPU rendering jobs to GPU nodes around the world on demand, providing a faster and cheaper ability to send token rewards to nodes after the creator has confirmed the rendering results. Setting up rendering infrastructure on-premises or adding GPU spending to cloud services requires a high upfront investment compared to traditional visual effects implementation methods.

Since its inception in 2017, Render Network users have rendered more than 16 million frames and nearly 500,000 scenes on the web. The data released from Render Network 2023 Q2 also shows that the number of render frame jobs and the number of active nodes are increasing. In addition, Render Network and 2023 Q1 have also launched a native integration with the Stability AI toolset, which allows users to introduce Stable Diffusion jobs to the AI field, and their business is no longer limited to rendering jobs but expands to the AI field.

Livepeer provides real-time video transcoding services for creators by contributing their own GPU computing power and bandwidth through network participants. Broadcasters can send videos to Livepeer to transcode various videos and distribute them to users on various devices, thereby disseminating video content. At the same time, you can easily pay in the form of fiat currency to obtain services such as video transcoding, transmission, and storage.

In the Livepeer network, anyone is allowed to earn fees by contributing personal computer resources (CPU, GPU, and bandwidth) to transcode and distribute video. The native token (LPT) represents the rights and interests of network participants in the network, and determines the weight of nodes in the network by staking tokens, thus affecting their chances of obtaining transcoding tasks. At the same time, LPT also plays a role in guiding nodes to complete the task of dispatching safely, reliably and quickly.

2. Expansion of the AI field

In the current ecosystem in the AI field, the main players can be broadly divided into:

Starting from the demand side, the demand for computing power is obviously different at different stages of the industry. Taking the underlying model development as an example, in the pre-training process, in order to ensure the effectiveness of the training results, the requirements for parallel computing, storage, and communication are very high, which requires a large computing power cluster to complete related tasks. At present, the main computing power supply mainly relies on self-built computer rooms and centralized cloud service platforms to provide centralized computing. However, in the follow-up model fine-tuning, real-time inference, and application development, the requirements for parallel computing and inter-node communication are not so high, which is precisely the part where decentralized computing power can show its strength.

Looking at the projects that have been quite vocal before, Akash Nework has made some attempts in the direction of decentralized computing power:

Akash Network combines different technology components to allow users to efficiently and flexibly deploy and manage applications in a decentralized cloud environment. Users can package applications using Docker container technology, which can then be deployed and scaled via CloudMOS over Kubernetes on cloud resources provided by Akash. Akash uses a “reverse auction” method, which makes the price lower than traditional cloud services.

Akash Network also announced in August this year that it will launch the 6th mainnet upgrade, incorporating GPU support into its cloud services to provide computing power to more AI teams in the future.

Gensyn.ai, this year’s high-profile project led by a16z has completed a $43 million Series A financing, according to the documents released so far, the project is a mainnet L1 PoS protocol based on the Polkadot network, focusing on deep learning, which aims to push the boundaries of machine learning by creating a global network of supercomputing clusters. This network connects everything from data centers with surplus computing power to PCs with potential personal GPUs, custom ASICs, and SoCs.

In order to solve some of the problems existing in the current decentralized computing power, Gensyn draws on some new theoretical research results from the academic community:

1. Adopt probabilistic learning proofs, i.e., use the metadata of gradient-based optimization processes to construct proofs of relevant task execution to speed up the verification process;

2. Graph-based Pinpoint Protocol, GPP serves as a bridge that connects the offline execution of DNN (Deep Neural Network) with the smart contract framework on the blockchain, which solves the inconsistencies that are prone to occur across hardware devices and ensures the consistency of verification.

3. Incentives similar to Truebit’s, through a combination of staking and penalties, build a system that allows economically rational participants to honestly perform their assigned tasks. The mechanism employs cryptography and game theoretic approaches. This validation system is essential for maintaining the integrity and reliability of large-scale model training computations.

However, it is worth noting that the above content is more about solving the task completion verification level, rather than the functions of decentralized computing power to achieve model training as the main highlight in the project document, especially about the optimization of parallel computing and communication and synchronization between distributed hardware. At present, due to the impact of network latency and bandwidth, frequent inter-node communication will increase the iteration time and communication cost, which will not only not bring actual optimization, but will reduce the training efficiency. Gensyn’s approach to node communication and parallel computation in model training can involve complex coordination protocols to manage the distributed nature of computation. However, without more detailed technical information or a deeper understanding of their specific approach, the exact mechanism by which Gensyn implements large-scale model training through its network will need to wait for the project to go live before it can be truly revealed.

We also pay attention to the Edge Matrix Computing (EMC) protocol, which uses blockchain technology to apply computing power to AI, rendering, scientific research, AI e-commerce access and other types of scenarios, and distributes tasks to different computing nodes through elastic computing. This method not only improves the efficiency of computing power, but also ensures the security of data transmission. At the same time, it provides a computing power marketplace where users can access and exchange computing resources. Developers can easily deploy and reach users faster. Combined with the economic form of Web3, it can also enable computing power providers to obtain real benefits and protocol subsidies based on the actual usage of users, and AI developers can also obtain lower inference and rendering costs. Here’s an overview of its main components and features:

It is expected that GPU-based RWA products will also be launched, the key to this is to revitalize the hardware that was originally fixed in the computer room, divide and circulate in the form of RWA, and obtain additional capital liquidity, the reason why high-quality GPU can be used as the underlying asset of RWA is that computing power can be regarded as hard currency in the field of AI, there is an obvious contradiction between supply and demand, and the contradiction cannot be solved in the short term, so the price of GPU is relatively stable.

In addition, the deployment of IDC equipment rooms to implement computing power clusters is also a key part of the EMC protocol, which not only enables the GPU to operate in a unified environment, but also more efficiently handles related large-scale computing power consumption tasks, such as model pre-training, so as to match the needs of professional users. At the same time, the IDC data center can also centrally host and run a large number of GPUs, ensuring the technical specifications of the same type of high-quality hardware, which is convenient to package and launch as RWA products to the market, opening up new ideas for DeFi.

In recent years, there have been new technical theories and application practices in the field of edge computing. As a supplement and optimization of cloud computing, edge computing is accelerating the speed from the cloud to the edge and into smaller and smaller IoT devices. These IoT devices tend to be small, so lightweight machine learning is favored to meet issues such as power consumption, latency, and accuracy.

Network3 is to provide services to AI developers around the world through AI model algorithm optimization and compression, federated learning, edge computing and privacy computing, helping them train or validate models quickly, conveniently and efficiently. By using a large number of intelligent IoT hardware devices, it can focus on small models to provide corresponding computing power, and by building a TEE (Trusted ution Environment), users can complete relevant training only by uploading model gradients to ensure the privacy and security of users’ related data.

In summary

• With the development of AI and other fields, many industries will achieve great changes from the underlying logic, computing power will rise to a more important position, and all aspects related to it will also cause extensive exploration in the industry.

• And the teams in the AI field themselves are also at a fork in the road, and the choice of whether to use the trained large models to build their own products or participate in the training of large models in their respective regions is also dialectical. Therefore, the development trend of decentralized computing power can meet different business needs, and with the update of technology and the iteration of algorithms, it is bound to make breakthroughs in key areas.

• Don’t be afraid, but take it slowly.