What I Built
I built FrameForge, an offline, AI-powered PC game settings optimizer designed for a friend who struggles with finding the perfect balance between frame rates and visual quality on their gaming rig.
PC gaming often involves tweaking dozens of graphics settings, which can be overwhelming and frustrating. FrameForge solves this by capturing real-time frame telemetry and automatically determining the optimal settings for smooth, stutter-free gameplay, completely offline.
Demo
You can check out the project landing page and download it here: [https://devanshindepth.github.io/frame-forge/]
Code
devanshindepth
/
frame-forge
Adaptive Game Graphics Optimizer
FrameForge — Adaptive Game Graphics Optimizer
Stop guessing graphics settings. Measure them.
FrameForge is a free Windows desktop application that measures your real frame rate while you play, tests graphics configurations automatically, and finds the fastest settings that preserve your chosen visual quality. Runs 100% offline on your own PC.
🚀 Downloads (v1.0.0)
| Package | Format | Architecture | Download Link |
|---|---|---|---|
| Windows Setup (Recommended) |
.exe (NSIS) |
x86_64 | FrameForge_1.0.0_x64-setup.exe |
| Windows Package |
.msi (WiX) |
x86_64 | FrameForge_1.0.0_x64_en-US.msi |
Integrity Verification (SHA-256)
# FrameForge_1.0.0_x64-setup.exe
71eeb4839a038c444b22ffcf06aa1d669c150226c5bee78479818a4cf01cfb18
# FrameForge_1.0.0_x64_en-US.msi
152695991ae4c052314779aff6bdd5625e5030416a0beee620acb3b34070930a
To verify on Windows PowerShell:
Get-FileHash -Algorithm SHA256 "path\to\FrameForge_1.0.0_x64-setup.exe"
✨ Features
- Real Frame-Rate Measurement: Measures average FPS and 1% lows using Intel PresentMon via Windows Event Tracing (ETW) — never injects into game memory.
- Safety First: Your game's configuration file is backed up before any test begins, and restored automatically if stopped, interrupted, or on application restart.
- Applied Verification: Confirms that the game actually…
How I Built It
FrameForge is built as a native Windows desktop application using Tauri (Rust core) with a vanilla HTML/JS/CSS frontend for a lightweight, snappy experience.
Its architecture consists of three components: a frontend for user interaction, a Rust-based core for game process monitoring, telemetry management, and safe configuration modification, and a Python AI sidecar packaged with PyInstaller. The Rust backend communicates with the AI engine through a bidirectional, line-delimited JSON IPC channel over stdin/stdout pipes. To ensure offline operation, the Python engine uses bundled TabPFN v2 model checkpoints, disables telemetry, and blocks outbound network connections, eliminating the need for cloud APIs or external inference services.
At the heart of FrameForge is TabPFN v2, an open-weight foundation model for tabular data, used as a surrogate model within an in-context Bayesian optimization pipeline. FrameForge captures real-time performance telemetry using Intel PresentMon (ETW) and converts graphics configurations into structured features, combining normalized settings with performance-cost priors and perceptual quality scores.
Below is the complete breakdown of results:
T1: Sample Efficiency (MAPE % vs. Sample Size n)
Lower MAPE is better.
| Model | n=8 | n=16 | n=32 | n=64 | n=128 | n=256 |
|---|---|---|---|---|---|---|
| TabPFN | 31.76% | 20.15% | 10.33% | 7.63% | 6.01% | 4.18% |
| GP (Matérn-5/2) | 33.87% | 28.60% | 13.77% | 8.30% | 6.90% | 5.00% |
| CatBoost | 30.88% | 22.02% | 14.18% | 8.96% | 7.71% | 6.71% |
| XGBoost | 34.67% | 23.21% | 15.86% | 11.34% | 8.14% | 7.03% |
| Random Forest | 30.35% | 26.19% | 20.73% | 12.64% | 9.56% | 7.64% |
| MLP | 32.01% | 26.75% | 18.62% | 15.59% | 11.40% | 9.55% |
T2: Constrained Bayesian Optimization Regret (% of Oracle Best)
Higher is better; tracks the percentage of optimal feasible objective reached.
| Optimiser / Surrogate | 5 evaluations | 10 evaluations | 15 evaluations | 20 evaluations |
|---|---|---|---|---|
| SMAC (Random Forest) | 20.0% | 20.0% | 56.1% | 60.5% |
| TabPFN-BO | 20.0% | 20.0% | 52.8% | 52.8% |
| XGBoost + Bootstrap EI | 20.0% | 46.0% | 47.1% | 51.1% |
| Random Search (Baseline) | 9.0% | 17.1% | 23.4% | 43.9% |
| GP-BO (BoTorch qEI) | 20.0% | 20.0% | 20.2% | 24.5% |
T3: Uncertainty Calibration (Empirical Coverage % vs Nominal)
Closer to Ideal coverage is better.
| Model | 50% Nominal | 80% Nominal | 90% Nominal | 95% Nominal |
|---|---|---|---|---|
| Ideal | 50.0% | 80.0% | 90.0% | 95.0% |
| Random Forest | 51.0% | 80.2% | 88.4% | 94.2% |
| TabPFN | 61.6% | 87.8% | 93.8% | 96.2% |
| GP (Matérn-5/2) | 38.5% | 60.1% | 68.3% | 73.8% |
| XGBoost | 31.0% | 54.2% | 63.8% | 70.2% |
| MLP | 26.1% | 48.0% | 58.6% | 66.5% |
| CatBoost | 7.2% | 13.8% | 18.2% | 21.6% |
T4: Measurement Noise Robustness (MAPE % vs Noise Level CV)
Lower MAPE is better; evaluated at n=32.
| Model | Noise CV=0.00 | CV=0.02 | CV=0.05 | CV=0.10 | CV=0.15 | CV=0.20 |
|---|---|---|---|---|---|---|
| TabPFN | 13.34% | 13.29% | 13.78% | 15.25% | 18.13% | 20.30% |
| XGBoost | 23.59% | 24.10% | 22.94% | 25.58% | 27.68% | 28.67% |
| CatBoost | 22.44% | 27.70% | 26.85% | 25.12% | 27.35% | 30.75% |
| MLP | 25.42% | 25.63% | 25.99% | 26.74% | 27.80% | 28.97% |
| Random Forest | 30.93% | 30.84% | 30.81% | 31.63% | 32.65% | 32.83% |
| GP (Matérn-5/2) | 19.27% | 17.50% | 21.47% | 26.22% | 31.21% | 35.58% |
T5: Cold Start / Transfer across GPUs (MAPE %)
Leave-one-GPU-out transfer with hardware features; lower MAPE is better.
| Model | Average FPS MAPE | 1% Low FPS MAPE |
|---|---|---|
| TabPFN | 14.14% | 23.69% |
| XGBoost | 18.50% | 24.46% |
| MLP | 18.81% | 24.09% |
| GP (Matérn-5/2) | 19.09% | 22.21% |
| CatBoost | 19.44% | 24.48% |
| Random Forest | 25.61% | 29.36% |
T6: 1% Low Ranking Ability (Spearman Rank Correlation)
Higher correlation is better; measures how accurately the model orders configurations.
| Model | Spearman |
|---|---|
| TabPFN | 0.870 |
| XGBoost | 0.799 |
| Random Forest | 0.783 |
| CatBoost | 0.736 |
| MLP | 0.725 |
| GP (Matérn-5/2) | 0.358 |
T7: Robustness to Missing Telemetry (MAPE % vs Missing Fraction)
Lower MAPE is better; hardware telemetry features randomly knocked out.
| Model | 0% Missing | 10% Missing | 20% Missing | 30% Missing | 40% Missing |
|---|---|---|---|---|---|
| TabPFN | 7.98% | 14.47% | 22.81% | 25.85% | 28.83% |
| XGBoost | 14.36% | 22.46% | 28.19% | 33.32% | 36.41% |
| Random Forest | 17.65% | 23.08% | 29.27% | 33.25% | 36.24% |
| CatBoost | 14.60% | 24.47% | 30.36% | 36.38% | 37.82% |
| GP (Matérn-5/2) | 11.99% | 20.40% | 28.42% | 32.47% | 39.00% |
T9: Compute Cost (Wall-Clock Seconds for n=64, 5,000 Candidates)
Lower wall-clock latency is faster.
| Model | Wall-Clock Latency |
|---|---|
| MLP | 0.27 s |
| GP (Matérn-5/2) | 0.39 s |
| Random Forest | 0.57 s |
| CatBoost | 6.71 s |
| XGBoost | 9.95 s |
| TabPFN | 30.35 s (CPU without CUDA) |
T10: Large-Data Scaling (MAPE %)
Scaling test from N=256 to N=1,000.
| Model | N=256 | N=1,000 |
|---|---|---|
| XGBoost | 14.23% | 7.95% |
| TabPFN | 10.75% | 8.00% |
| CatBoost | 11.75% | 8.20% |
The optimizer generates up to 4,000 candidate configurations per iteration, filters out those that violate the user's quality requirements, and uses TabPFN's quantile predictions to estimate performance and uncertainty. An Expected Improvement acquisition function guides the search, balancing average FPS, 1% lows, and visual quality against the target refresh rate. Optimization progresses from baseline measurements and exploratory sampling to active AI-guided search, with early stopping when further improvements become unlikely.
Why Does Open Innovation Matter?
Open innovation matters because it ensures privacy, accessibility, and user control. For FrameForge, it was critical that the optimization process ran entirely offline on the local machine without sending hardware telemetry or gaming habits to the cloud.
Using an open-weight model like TabPFN made this local execution possible. A closed API would have introduced latency, required a constant internet connection, and compromised user privacy. Furthermore, open-source tools like Tauri and PresentMon provided the reliable, high-performance foundation needed to build this seamlessly.
Prize Categories
TabPFN Challenge - The benchmark results strongly validate that choosing TabPFN was the right decision for this project.
Top comments (1)
forgot to add these instructions: