DEV Community

#vllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Serverless Gemma 4: E2B on a Lambda GPU Function, with Google's QAT Weights and Drafter

Serverless Gemma 4: E2B on a Lambda GPU Function, with Google's QAT Weights and Drafter

15
Comments
10 min read
vLLM vs Ollama: which should you use to serve an LLM?

vLLM vs Ollama: which should you use to serve an LLM?

Comments
10 min read
vLLM 0.34 Stops Reserving 8% of Your GPU: Extensible KV Cache, the Block Size 64 Default, and What to Delete From Your Launch Flags

vLLM 0.34 Stops Reserving 8% of Your GPU: Extensible KV Cache, the Block Size 64 Default, and What to Delete From Your Launch Flags

Comments
14 min read
How do you host your own LLM behind an OpenAI-style API?

How do you host your own LLM behind an OpenAI-style API?

Comments 1
9 min read
Inside vLLM's Scheduler: Continuous Batching, Chunked Prefill, and Who Waits

Inside vLLM's Scheduler: Continuous Batching, Chunked Prefill, and Who Waits

Comments
10 min read
Gemma 4 on AWS Lambda with a GPU: E2B on an NVIDIA L4, Day One

Gemma 4 on AWS Lambda with a GPU: E2B on an NVIDIA L4, Day One

10
Comments
10 min read
Tuning vLLM on NVIDIA vGPUs

Tuning vLLM on NVIDIA vGPUs

Comments
8 min read
llm-d v1.0 on Kubernetes: KV-Cache Routing, Disaggregation, and Production Readiness

llm-d v1.0 on Kubernetes: KV-Cache Routing, Disaggregation, and Production Readiness

Comments
10 min read
Serving 753B MoE Across Nodes: vLLM & RDMA Limits

Serving 753B MoE Across Nodes: vLLM & RDMA Limits

Comments
1 min read
Day 5: Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution

Day 5: Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution

Comments
3 min read
Prefill vs Decode: Why Splitting Them Wins

Prefill vs Decode: Why Splitting Them Wins

1
Comments 1
10 min read
vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade

vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade

Comments
9 min read
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff

llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff

Comments
9 min read
Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack

Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack

1
Comments 1
15 min read
verl GRPO + vLLM Rollouts: GPU Memory Settings for Single-Node Deployment

verl GRPO + vLLM Rollouts: GPU Memory Settings for Single-Node Deployment

Comments
12 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.