Agent configs and billing exports often contain secrets: API keys, account IDs, internal service names. So when I built Arup Banerjee Labs, I set one rule first: whatever you paste stays in your browser.
The result is three small, free tools that turn a paste into a scorecard.
The constraint that shaped the suite
Most "analyze your config" tools want you to upload a file or grant OAuth access to your cloud org. That's a big ask when all you want is a quick sanity check. A static site that parses and scores locally removes the trust problem: there's no backend to leak to, because there's no backend.
Everything is hosted on GitHub Pages, and the source is public: https://github.com/ultimatixarup/arup-banerjee-labs
Tool 1: Agent Health Checker
Paste an AI agent config (JSON, YAML, or plain text) and get a reliability/privacy scorecard. It looks for the things demos tend to hide:
- retries and backoff
- timeouts
- tracing hooks
- secrets sitting in the config
- tool-auth notes and injection-prone tool sinks
- evals/tests
- a single LLM with no fallback
https://ultimatixarup.github.io/arup-banerjee-labs/agent-health-checker/
Tool 2: Cloud Bill Smell Detector
Paste a cost CSV, invoice export, or billing JSON that you exported from your own account, and get an educational smell scorecard: idle-looking lines, missing tags, transfer-heavy patterns, and similar heuristics.
Explicit non-goals: it never connects to AWS, GCP, or Azure, and it only looks at text you paste.
https://ultimatixarup.github.io/arup-banerjee-labs/cloud-bill-smell/
Tool 3: Observability Gap Finder
Paste OpenTelemetry, Prometheus, or Splunk-style config snippets, or a service inventory YAML, and get a gaps scorecard: missing SLIs, services without tracing, no cardinality guards.
https://ultimatixarup.github.io/arup-banerjee-labs/observability-gap-finder/
What "heuristic" means here
These are prioritization aids, not audit stamps. They'll produce false positives, and they won't catch everything. The goal is a fast "what should I look at first?" before something hits production.
Verify the privacy claim yourself
Open your browser's DevTools, switch to the Network tab, paste something, and run a score. Your paste should never leave the page. If you ever see it go out, that's a bug and I want to know.
What's next
I'm collecting feedback on wrong and missing checks. If one of these flags something silly, or misses something obvious in your stack, open an issue on the repo or leave a comment here.
Try the suite: https://ultimatixarup.github.io/arup-banerjee-labs/
Arup Kumar Banerjee · Little Elm, Texas
Questions? Message my free help bot on Telegram: t.me/ArupLabsHelpBot_bot
Personal project built on my own time. Not affiliated with or endorsed by my employer; views are my own.
Top comments (9)
the devtools network tab test is the strongest part of this. "we never see your data" is usually a vibe, making it verifiable is what makes it real.
of the three, the agent health checker is the one i'd reach for first. secrets-in-config and injection-prone tool sinks are the checks that actually bite people, the rest is nice to have.
one thing i'd think about: if the feedback loop for wrong checks lives in github issues, someone will eventually paste a config with a live key into a "this flagged nothing" report. the privacy story is airtight on the tool side, the report side is the leakier surface.
Thanks Om, this is really useful. Agreed that secrets-in-config and injection-prone tool sinks are the checks that bite. And you're right, the report side is the leakier surface. I'm going to add a clear "redact keys and tokens before you paste" warning to the GitHub issue template so nobody drops a live key into a report.
nice, glad that landed. the template warning is exactly the fix i'd have suggested, catches people at the point of paste instead of after. though i'd bet the next leakier-than-expected surface is people pasting their scorecards into a chatbot to "explain my bill". one hole at a time.
Ha, good call. The scorecard Markdown is meant to stay with you — pasting it into a chatbot to explain a bill reopens the privacy hole. One hole at a time.
yeah, we're converging on the same fixes. the annoying bit is the ordering: people do the careful local thing, then paste the raw scorecard into a chatbot for "explain my bill". the template warning catches github reports, a one-liner in the scorecard itself would catch the artifact that actually travels.
Yep — the issue template catches the report path, but the Markdown export is what people actually carry around. I'll put a one-liner on the scorecard itself so the warning travels with the artifact.
Shipped. Copy Markdown on all three tools now opens with a one-line note to redact secrets, account IDs, hostnames and customer data before sharing or pasting into a chatbot. Thanks for pushing on this, Om.
love that it shipped. a one-line nudge right where the export happens beats a warning paragraph people scroll past.
Agreed — right at the export is where people actually look. Glad it landed.