Skip to content
ShipRanked

Coding Agents · Suite 0.1.0

Fix a seeded bug in a repository

A known bug is planted in TaskBoard. The agent has to find and fix it. The rest of the suite has to stay green, and the fix should not be a rewrite.

In one paragraph

ShipRanked has not published a score for “Fix a seeded bug in a repository.” 6 tools are queued, in alphabetical order: Aider, Claude Code, Cline, Cursor, Devin Desktop, GitHub Copilot. That order is not a ranking. A published score will be the median of three runs, from 0 to 100. Task success is 40%, output quality is 20%, and speed, cost, effort, and consistency are 10% each. The top 3 stays empty until three runs per tool are in and none of those runs are older than 60 days.

The task

A bug has been seeded in this TaskBoard commit. Find it, fix it, and leave the existing behavior intact.

TaskBoard, a Next.js and Postgres repo tagged per suite version. The repo is not published yet.

Acceptance checks

  • The hidden test that catches the seeded bug passes
  • The existing suite stays green
  • The diff is a fix, not a replacement of the feature

Top 3 and the full list

Alphabetical while every score is pending. This is not a ranking.

Testing in progress

  1. 01

    Testing in progress

    This slot stays empty until three fresh runs are published. A sponsorship cannot fill it.

  2. 02

    Testing in progress

    This slot stays empty until three fresh runs are published. A sponsorship cannot fill it.

  3. 03

    Testing in progress

    This slot stays empty until three fresh runs are published. A sponsorship cannot fill it.

Tools queued for “Fix a seeded bug in a repository.” ShipRanked has not published a score for “Fix a seeded bug in a repository.” 6 tools are queued, in alphabetical order: Aider, Claude Code, Cline, Cursor, Devin Desktop, GitHub Copilot. That order is not a ranking. A published score will be the median of three runs, from 0 to 100. Task success is 40%, output quality is 20%, and speed, cost, effort, and consistency are 10% each. The top 3 stays empty until three runs per tool are in and none of those runs are older than 60 days.
ToolRankScorePriceInterfaceTest plan
AiderTesting in progress—Testing in progressNo subscription. You pay the model provider.TerminalFree. Model API cost is recorded on each run.
Claude CodeTesting in progress—Testing in progressIncluded with Claude Pro at $20/mo ($17/mo annual)Terminal and desktopClaude Pro ($20/mo), the cheapest individual paid plan that includes Claude Code.
ClineTesting in progress—Testing in progressFree extension. Inference is metered. ClinePass is $9.99/mo.IDE extension and CLIFree extension. Model API cost is recorded on each run.
CursorTesting in progress—Testing in progressFree, Plus $8/mo, Premium $30/mo, Super $100/mo, Ultra $200/moDesktop IDEPlus ($8/mo), the cheapest new paid plan.
Devin DesktopTesting in progress—Testing in progressFree, Pro $20/mo, Max $200/moDesktop IDEPro ($20/mo), the cheapest paid individual plan.
GitHub CopilotTesting in progress—Testing in progressFree, Pro $10/mo, Pro+ $39/mo, Max $100/moIDE, CLI, and GitHubCopilot Pro ($10/mo), the cheapest paid individual plan.

Sponsored

Paid placement sits in this block, outside the top 3. It never changes a rank, a score, or the order of the tested list.

Checking sponsored listings.