DEV Community

#benchmarking

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Why my first CPU benchmark was too noisy to use, and how I fixed it

Why my first CPU benchmark was too noisy to use, and how I fixed it

Comments
3 min read
The 1/20-cost model scored less than half the coding points — so I made the 60-second proof public

The 1/20-cost model scored less than half the coding points — so I made the 60-second proof public

1
Comments
1 min read
New 'Flashwright' Software Testing Framework Faces Unverified Claims and Trademark Scrutiny

New 'Flashwright' Software Testing Framework Faces Unverified Claims and Trademark Scrutiny

Comments
8 min read
86.5 beats Opus on my own laptop — the Shorts walkthrough is up

86.5 beats Opus on my own laptop — the Shorts walkthrough is up

1
Comments
1 min read
99.98% blocked: I ran 9,000 Shorts through my filter so the old model could fail in peace

99.98% blocked: I ran 9,000 Shorts through my filter so the old model could fail in peace

1
Comments
1 min read
I built an independent clock to time Gemini 4 Hydrogen — the 2.5x claim vs my 1.1x, and what it cost me

I built an independent clock to time Gemini 4 Hydrogen — the 2.5x claim vs my 1.1x, and what it cost me

1
Comments
1 min read
Your Private Eval Is the Only Benchmark That Steers You

Your Private Eval Is the Only Benchmark That Steers You

Comments
7 min read
Same quantized model, two different fates: one chip refused it, the other ran it in 1.39 ms

Same quantized model, two different fates: one chip refused it, the other ran it in 1.39 ms

Comments
3 min read
How an LLM Benchmark Produces Comparable Numbers: Inside LFORLA's LLM Evaluation

How an LLM Benchmark Produces Comparable Numbers: Inside LFORLA's LLM Evaluation

Comments
3 min read
My A/B rig had 95% confidence intervals and was wrong in both directions

My A/B rig had 95% confidence intervals and was wrong in both directions

Comments
9 min read
LLM Evaluation, Explained: How a Benchmark Produces Comparable Numbers (TL;DR)

LLM Evaluation, Explained: How a Benchmark Produces Comparable Numbers (TL;DR)

Comments
4 min read
The agent called the framework's MCP server in 0 of 5 runs

The agent called the framework's MCP server in 0 of 5 runs

Comments
6 min read
How to install Siege on Ubuntu ( The Most Advanced Benchmarking Tool)

How to install Siege on Ubuntu ( The Most Advanced Benchmarking Tool)

Comments
4 min read
LLM Evaluation That Actually Compares: Inside the FreeCAD Fix Benchmark

LLM Evaluation That Actually Compares: Inside the FreeCAD Fix Benchmark

Comments
4 min read
FP8 Cut Our TPOT by 25%. At High Concurrency, It Also Changed the Answers.

FP8 Cut Our TPOT by 25%. At High Concurrency, It Also Changed the Answers.

Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.