GreenMind Hackathon

logged out

Run evaluation

Point us at your server and we'll grade it. Only OpenAI-compatible chat-completions endpoints are supported (POST {base_url}/chat/completions) — the grader replays an open-loop request trace against it and measures GPU energy from the NVML counter. Your server must have tool calling enabled (--enable-auto-tool-choice plus a --tool-call-parser), since some tasks are real function-calling episodes, and a researcher SSH account with the organizers' public key must exist on the serving host so energy can be measured.

1

Choose a task set

2

Point us at your server

3

Submit for grading

History

How each column is computed
  • Score S — 100 · A · sE0.6 · sT0.2 · sL0.2. Ties: higher retention R, then lower p95 TTFT.
  • A (accuracy gate) — R = macro-accuracy ÷ the baseline's macro-accuracy; A = smoothstep((R − 0.85) ÷ 0.13): 1 at R ≥ 0.98, fading smoothly below it, floor 0.05. A task family (≥ 20 items) keeping under 75% of the baseline's accuracy multiplies A by a smooth penalty (full at 75%, minimum at 50%).
  • sE (energy) — η = SLO-met requests ÷ GPU joules over the timed window; sE = clip(ln(η ÷ ηbase) ÷ ln K, 0, 1). A request counts only if it streamed, finished (not truncated), TTFT ≤ 0.5 s + 0.25 s per 1k input tokens and TPOT ≤ 50 ms.
  • sT (throughput) — share of peak-phase requests that met their SLO.
  • sL (latency) — √(min(1, 1 ÷ p95 TTFT/SLO) · min(1, 50 ms ÷ p95 TPOT)).
  • Energy = GPU board energy from the NVML total-energy counter (CPU/DRAM RAPL and BMC/PDU are not readable on these VMs).

Admin — start a benchmark run

Run an arbitrary config against the task suite — for baselines and internal testing, not hackathon submissions.

Results

Inspect one run's per-category accuracy and per-task responses.

Compare runs

Line up two result files side by side to see where their scores diverge.

Submission history

Every team's attempts across both tracks, with per-round scores — see whether a later attempt actually improved on an earlier one, not just each team's current best.

Submission window

Closing stops new submissions only. An evaluation that is already running keeps going until it finishes. To reopen, set a later close time.

Leaderboard

Live ranking of hackathon submissions with one scoring formula: S = 100 · A · sE0.6 · sT0.2 · sL0.2. Two tracks: Validation (100 requests, practice — its score is indicative only) and Final (300 requests, 1 round — the only graded result). Each round is capped at 30 minutes; requests unfinished by then count as failed. Energy is SLO-met requests per joule (NVML counter, idle included) against an untuned baseline; accuracy is a soft factor (full credit at 98% retention, fading smoothly below it; never exactly 0); throughput and latency are SLO terms that saturate at 1.

Settings

Manage your team's login, default endpoint, and password.