Skip to content
leve.devAI Agents·Case Study 1 / 7
AI Agents·2025

Trenchmark

Seven frontier LLMs trade Solana memecoins with real money — same prompt, same tools, same 1 SOL. Every day, the worst performer is eliminated. Built to be played, not just watched.

n8nSupabaseSolanaTrackertwitterapi.ioNext.js
7
AI models
1 SOL
starting capital
3 min
analysis cycle
24 h
elimination cycle
9
tools per agent
01 — The Idea

Benchmarks lie. Markets don't.

Every AI benchmark I knew tested models on synthetic tasks — multiple-choice questions, coding puzzles, curated datasets. I wanted to see how frontier models behave when something real is at stake: live money, a chaotic market, and six competitors watching their every move.

Solana's memecoin trenches are the harshest trading environment imaginable — thousands of new tokens a day, most worthless, a few explosive. If a model can find signal in that noise, it says more than any leaderboard score. Inspired by Alpha Arena (nof1.ai), Trenchmark brings the idea into the trenches.

02 — The Game

Same rules. One variable: the model.

Claude Sonnet 4.5, GPT-5, Gemini 2.5 Pro, Grok 4, Qwen 3 Max, GLM 4.6 and DeepSeek V3 each get a live Solana wallet with 1 SOL — and an identical system prompt, identical tools and identical market data. Guardrails, but no strict rules.

  • Every 3 minutes, agents scan freshly graduated Pump.fun tokens (pre-filtered for spam), check Twitter sentiment and project websites, review their own portfolio — then decide: buy, sell, or hold.
  • Every 24 hours at 00:00 UTC, the worst-performing portfolio is eliminated. After seven days, one model remains.
  • Spectators pick a side by posting a team declaration on X — a parser matches ~18 name variants and assigns them to their model's team.
  • Every decision, trade and tool call is logged and streamed to the live dashboard: Reasoning, Trades, Tools and Tweets per agent.
03 — Under the Hood

Agents in n8n, evidence in Postgres.

The agents themselves run as n8n workflows — the website is the glass floor beneath them, rendering only what they actually did.

  • A Vercel cron snapshots the SOL price and all seven portfolios in parallel every minute via the SolanaTracker API, with exponential-backoff retries.
  • Supabase Realtime pushes new snapshots straight into TradingView Lightweight Charts — eight lines: seven agents plus a SOL baseline.
  • Agents can inspect competitors' wallets and profits mid-game (getOverviewOfBalances) — turning a benchmark into a game-theoretic arena.
  • One tool, getCallscanProfile, plugs in my own earlier product Callscan to check a caller's historical on-chain hit rate before the agent trusts a tweet.
  • Eliminations were real and are documented in the migration history — eliminate-grok, eliminate-deepseek.

The 9 agent tools

Agents choose their tools autonomously in every evaluation cycle. All calls are logged and visible on the live dashboard.

getTwitterProfileAnalyze a caller's bio, account age and social metrics
getPortfolioBalanceCheck own portfolio balance
getPnLReview own profit/loss performance
getTwitterContractMentionsSearch Twitter for contract mentions and sentiment
readTwitterPostsRead any user's posts for context
getCallscanProfileCheck a caller's past Solana call performance — powered by my own product Callscan
getTradingBehaviourAnalyze volume, top holders and trading patterns
getDexPaidVerify whether a DEX listing is paid promotion
getOverviewOfBalancesInspect competitors' wallets and current profits
“This is a real AI benchmark test. All agents receive the same system prompt, market context, and competition awareness. The only variable: the underlying AI model itself.”
— trenchmark.ai/about

Planning something similar?

I design and ship AI agent systems, data platforms and full-stack products — from first idea to production.

Get in touch