DEV Community

#llminference

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Is your LLM speedup real? Test it with interleaved controls

Is your LLM speedup real? Test it with interleaved controls

Comments
7 min read
How would you balance this trade-off in practice?

How would you balance this trade-off in practice?

Comments
2 min read
When a single GPU isn't sufficient to host or compute the model, how would you split a large LLM across multiple GPUs or…

When a single GPU isn't sufficient to host or compute the model, how would you split a large LLM across multiple GPUs or…

Comments
2 min read
H200 vs H100 for LLM inference: which GPU to buy, and where the B200 fits

H200 vs H100 for LLM inference: which GPU to buy, and where the B200 fits

Comments
13 min read
FreeToken: Run Frontier MoE Models on Your Own GPU, and the RAM It Takes

FreeToken: Run Frontier MoE Models on Your Own GPU, and the RAM It Takes

Comments
9 min read
What optimizations would you employ to achieve high throughput and low latency?

What optimizations would you employ to achieve high throughput and low latency?

Comments
2 min read
GLM-5.3 leaked. Is the API relay you're using safe?

GLM-5.3 leaked. Is the API relay you're using safe?

Comments
5 min read
LLM cold starts on HyperPod: from 27 minutes to seconds with model caching

LLM cold starts on HyperPod: from 27 minutes to seconds with model caching

Comments
9 min read
vLLM v0.28.0: the breaking change small GPU users must read

vLLM v0.28.0: the breaking change small GPU users must read

Comments
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.