Howdy!

For me, the game theory between validators is one of the most interesting parts of crypto. Today’s main item is a bit long, but it’s worth your time, I promise.

Today, we’ve got a deep dive on Solana’s slow validator problem and an update on Pump.fun’s token buybacks:

Behind the scenes of Solana’s slow validator problem

As issues with Solana’s code have been ironed out over the years, block times — or how long it takes the network to produce a new block of transactions — have declined, even dipping under Solana’s putative level of 400ms.

But over the past month, an interesting trend has emerged: Median block times spiked, and Solana was adding new transactions to the blockchain slower than before. At fault was a new meta among Solana validators that showed it can be lucrative to produce blocks slowly. Anza, Jito and Marinade appear to be weighing fixes to the problem, Blockworks has learned.

Every Solana block has a validator who serves as leader — meaning they collect transactions, create a block and broadcast it to the network. Leaders collect transaction fees from the blocks they create. More order flow means more fee opportunities, so it can be more lucrative for validators to see and process 500ms worth of transactions as opposed to 300ms, for instance. 

At a basic level, some Solana validators appear to be waiting as long as possible to pack blocks with transactions in hopes of maximizing their yield, which has caused Solana’s epoch lengths to increase. 

This isn’t ideal for a network focused on being as fast as the Nasdaq. Plus, fewer epochs per year means fewer opportunities for staking rewards to compound for stakers, Sol Strategies chief technology officer Max Kaplan noted.

Solana offers something called “grace ticks,” which is a late period wherein leaders can still successfully submit blocks. This prevents validators in far-flung locations from being unfairly penalized, but it also opens the door for validators to be late on purpose. 

The alternative Solana client Frankendancer also recently released a revenue-maximizing scheduler, and validators running the client appear to be packing their blocks a bit more slowly than normal, Kaplan said. 

Kaplan added that the Frankendancer delay is minimal compared to more egregious delayers, and that he wouldn’t portray it as a “bad thing.” Plus, block delaying isn’t a new concept on proof-of-stake blockchains. Firedancer’s upgrade may have brought this strategy to light on Solana, however. Jump did not immediately return a request for comment.

Interestingly, Firedancer software engineer Michael McGee described the phenomenon on this week’s Lightspeed podcast episode.

“One of the things we've seen with our current validator…[is that validators] can often make a more profitable block by delaying execution of transactions,” McGee noted.

Solana validators who are more noticeably delaying blocks tend to be running modified versions of the Agave-Jito client, Blockworks Research analyst Victor Pham noted. 

For example, during epoch 802 in mid-June, Galaxy and Kiln’s median block time was higher than 570ms each. A number of unlabeled validators were also slow, and Temporal’s validators had 475ms median block times, according to Solana Compass data.

Kiln co-founder Ernest Oppetit admitted the validator — Solana’s sixth-largest by stake — had been delaying slots “for a period” but said it has stopped doing so.

“At Kiln we pride ourselves in offering the top staking APY in the market without compromising on security. We have been doing R&D on different parts of the stack, including timing games, and are in constant discussions with our customers, client teams and the foundation on this. At this point we are following the spec and not delaying blocks, but many actors are still doing so. We believe the incentive issue (fast block production leads to lower rewards) ultimately needs to be addressed at the protocol level,” Oppetit said.

“I will say that we are not the reason that people know about this,” Temporal engineering director Ben Coverston said when reached for comment about the validator’s apparent participation in the slow block trend.

“We, as a service provider, support validator configurations that prioritize maximizing staking rewards for our clients. On Solana, that may mean proposing slightly slower blocks to ensure higher reward capture. Galaxy also remains responsive to community feedback and have since increased block times to within an agreeable threshold,” a Galaxy spokesperson said.

The Solana validator community doesn’t find slowing the network down to be kosher, and slow validators are now facing public backlash.

They may soon face more material punishments. Blockworks has learned that Jito plans to blacklist slow validators from its stake pool, which is Solana’s largest.

Jito Foundation president Brian Smith said the organization is “drafting a governance proposal that empowers a committee to remove laggards from the JitoSOL delegation set. This should be available for community discussion within a few days.”

Michael Repetny, co-founder of the third-largest stake pool in Marinade, said the pool provider is “[considering] putting it to a governance proposal to discuss pros and cons of introducing [slow validators] as a hard rule/offense of the delegation strategy.”

Help is on the way at the protocol level, too. Anza’s GitHub repository shows a new proposal to shrink Solana’s grace tick period by half. Plus, Solana’s proposed consensus overhaul should fix things.

“Alpenglow will solve this by enabling skip votes,” Anza vice president of core engineering Brennan Watt said.

Watt said on a recent Lightspeed podcast episode that Anza hopes to have Alpenglow live on mainnet by Solana’s Breakpoint conference in December. 

— Jack Kubinec

P.S. Fill out our short audience survey and help us build a better Lightspeed. Thank you!

In crypto, hype fades but sustainable ecosystems win. Learn how top protocols are building for the long haul.

Join Blockworks Research, MegaETH, Flipside Crypto and Avalanche for this upcoming Roundtable.

📅 July 31 | 12 PM EST

Here’s why Solana block times slowed down:

This data from Staking Facilities’ Matthias Schmitz shows the revenue of a slow validator vs. a fast one.

The orange (slow) validator outperforms the blue (fast) validator most of the time. That’s a problem if the goal is to IBRL.

— Jack Kubinec

Just a day after Pump.fun’s ICO, the team kicked off a token buyback wherein it used treasury funds to absorb nearly 3 billion $PUMP from the open market.

The buyback concept has gained traction in recent months as a way for crypto platforms to reinvest revenue, support token value and reward long-term holders.

In Pump.fun’s case specifically, the platform generates millions in fees from launches and trading activity. Instead of leaving that SOL to idle, Pump has opted to buy back its own token from the market, effectively injecting value back into the ecosystem.

We reported before the sale that Pump.fun committed to routing 25% of platform revenue back to holders. After pausing for a day, the buyback wallet resumed activity on July 17, with more transactions ongoing at the time of publication. The buyback wallet currently sits 16% underwater, reflecting a net loss of approximately $2.9 million.

— Jeffrey Albus

A message from Max Kaplan, chief technology officer at Sol Strategies: