Polkadot
NEAR Protocol
The Scalability Trilemma
Every blockchain designer confronts a fundamental constraint: you can optimize for two of three properties — decentralization, security, and scalability — but achieving all three simultaneously is extraordinarily difficult. This tension, often called the scalability trilemma, explains why early blockchains like Bitcoin and Ethereum process only a handful of transactions per second while Visa handles tens of thousands.
Sharding is one of the most ambitious attempts to escape this trilemma. Rather than forcing every node to process every transaction, sharding divides the network into parallel subgroups — shards — each responsible for a fraction of the total workload. Done correctly, a sharded blockchain's throughput scales roughly linearly with the number of shards, while each individual node carries a manageable, bounded load.
Three Dimensions of Sharding
Sharding is not a single technique but a family of related approaches. Understanding the distinctions matters because each type has different security implications.
Network Sharding
The simplest form divides nodes into subgroups, with each subgroup communicating primarily within itself. This reduces the networking overhead for any individual node. However, if a shard's node set is too small, an attacker controlling a significant fraction of total network stake might concentrate their resources to corrupt a single shard — a phenomenon called a single-shard takeover attack.
The solution is random shard assignment, rotated frequently, so no attacker can predict or accumulate stake in a target shard before the rotation occurs. Ethereum's beacon chain design uses RANDAO (a randomness scheme based on validator-contributed entropy) plus VDF (Verifiable Delay Functions) to achieve unpredictable, unbiasable committee assignments.
Transaction Sharding
Transaction sharding assigns transactions to shards based on their properties — typically the sender's address. Each shard processes its assigned transactions independently. This works well when transactions are self-contained (Alice sends Bob tokens, where both are in the same shard) but becomes complicated when transactions span shards.
Cross-shard transactions require a communication protocol. One approach is two-phase commit: the originating shard locks the sender's funds, sends a cross-shard message to the receiving shard, and the receiving shard either commits or aborts. If the receiving shard aborts, the originating shard unlocks the funds. This adds latency proportional to the number of cross-shard hops.
state-sharding">State Sharding
State sharding is the most complete and most challenging form. Each shard maintains only a portion of the global state — the account balances, smart contract storage, and code for addresses assigned to that shard. No single node holds the complete state of the network.
This dramatically reduces per-node storage requirements as the network grows, but it introduces the hardest problem in sharding: how does Shard A validate a transaction from Shard B if Shard A doesn't have Shard B's state? Cross-shard communication must carry not just messages but also cryptographic proofs that the referenced state is valid.
Cross-Shard Communication
Cross-shard communication is the central engineering challenge of sharded blockchains. Every design makes trade-offs between latency, complexity, and security.
Ethereum's research into sharding (documented in the Ethereum research forums and incorporated into the beacon chain design) settled on an asynchronous model. Shards produce "blob" data that gets attested to by committees of validators. The beacon chain acts as a coordination layer, recording shard commitments and ensuring that cross-shard messages are eventually delivered.
Polkadot's whitepaper takes a different approach. Its relay chain plays the role of Ethereum's beacon chain, but Polkadot's "parachains" (parallel chains) are more autonomous. Cross-chain message passing (XCMP) allows parachains to communicate directly rather than routing everything through the relay chain, reducing latency for common cross-chain interactions.
Near Protocol's "Nightshade" sharding design merges all shards into a single logical chain where each block is actually composed of shard "chunks." Validators assigned to a shard produce chunks, and block producers assemble them. This simplifies the cross-shard model by maintaining a single global block production process.
Data Availability Sampling: The Key Enabler
One of the most elegant solutions to emerge from sharding research is data availability sampling (DAS). It addresses a fundamental problem: in a sharded system, how do light nodes verify that all shard data was actually published, without downloading everything?
The answer uses erasure codes. When a shard block is produced, it is encoded with a 2D erasure code that expands it to four times its original size. Any 50% of the coded data is sufficient to reconstruct the original. A light node can randomly sample a small number of individual coded chunks. If all sampled chunks are available and valid, the light node gains high statistical confidence that the full data is available — because if a significant fraction of data were withheld, random sampling would almost certainly catch missing chunks.
This is the design at the heart of Ethereum's danksharding roadmap. Proto-danksharding (EIP-4844) introduced "blobs" — large data packets attached to blocks but not executed by the EVM — as a stepping stone. Full danksharding will extend this with DAS, allowing nodes to verify data availability without downloading blobs in full.
Security Considerations
Sharding introduces security risks that monolithic chains do not face.
Adaptive corruption is the risk that an attacker, given enough time, can identify which validators are assigned to a vulnerable shard and bribe or compromise them before reassignment. Random frequent reshuffling is the primary defense.
Cross-shard fraud requires that when state from one shard is referenced in another, the receiving shard can verify the referenced state's validity. Fraud proofs allow any validator to challenge an invalid state transition after the fact; validity proofs (ZK proofs) allow verification before acceptance. The industry is moving toward validity proofs for stronger security guarantees.
Data withholding attacks occur when a block producer publishes a block header but withholds the underlying data. Other validators cannot verify the block's contents and cannot generate fraud proofs for invalid transactions. DAS is specifically designed to detect this attack.
The State of Sharding Today
Full state sharding at production scale remains an unsolved engineering challenge as of 2024. Ethereum has made the most systematic progress with its phased roadmap: the beacon chain (launched 2020) established the validator set and randomness infrastructure; the Merge (2022) transitioned consensus to proof of stake; EIP-4844 (2024) introduced blob data for rollup scaling; full danksharding remains on the horizon.
Practical scaling has been largely achieved through a complementary approach: Layer 2 rollups, which execute transactions off-chain and post compressed proofs or data to the base layer. Sharding the base layer's data availability (rather than its execution) turns out to be sufficient to support enormous Layer 2 throughput — this insight reshaped Ethereum's scaling strategy.
The intellectual legacy of sharding research, however, extends beyond any single implementation. The concepts of committee-based consensus, data availability sampling, and fraud proofs have influenced the design of nearly every high-throughput blockchain built in the past five years. Sharding remains the most theoretically principled path to a blockchain that is simultaneously decentralized, secure, and capable of serving global financial infrastructure.