Compare several models' answers to your own prompt without seeing which is which
Intermediate12h1–2
Vercel AI SDKNext.jsSQLite
17/20#testing hackathon ideas
13 ideas across 4 categories · 1 beginner · 8 intermediate · 4 advanced. Sorted by judging fit (editorial estimate).
Ideas tagged testing
Devtools
Regex from Examples
Give matching and non-matching examples and get a tested regex
Beginner8h1LLMJavaScript17/20Devtools
Visual Regression Bot
Screenshot key pages on each PR and highlight visual changes
Intermediate16h1–2PlaywrightGitHub ActionsNode.js17/20Devtools
Unit Tests from Diffs
Generate missing unit tests for the code changed in a PR
Intermediate16h1–2LLMNode.jsVitest17/20Devtools
MCP Server Inspector
List, call and test an MCP server's tools with schema checks
Intermediate14h1MCPTypeScriptNext.js17/20Devtools
Prompt Regression Tests
Catch quality drops when prompts or models change
Intermediate14h1–2Node.jsLLMGitHub Actions17/20Attack your own LLM app with known injection patterns and see which ones get through
Intermediate18h1–3LLMNode.jsNext.js16/20Generate realistic fake data with the same shape and stats as your real data
Intermediate16h1–2PythonDuckDBLLM16/20Gaming
Automated Playtester Bot
A bot that plays your web game and reports bugs and difficulty spikes
Advanced24h1–2PlaywrightNode.jsECharts16/20AI Agents
Dependency Upgrade Agent
An agent upgrades a dependency, fixes breaking changes and opens a PR with test results
Advanced28h1–3LLMGitHub APINode.js15/20AI Agents
Exploratory QA Agent
An agent clicks through your web app like a curious user and reports bugs
Advanced28h1–3browser-usePlaywrightLLM15/20Devtools
Flaky Test Detector
Identifies tests that randomly fail and why
Intermediate24h1–3PythonJUnit XMLReact15/20AI Agents
Bug Report to Repro Script Agent
Paste a vague bug report and the agent tries to reproduce it as a Playwright test
Advanced30h2–3PlaywrightLLMLangGraph14/20