Skip to content
Navigation menu
Search
Powered by Algolia
Search
Log in
Create account
DEV Community
Close
#
vllm
Follow
Hide
Posts
Left menu
đź‘‹
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
Serverless Gemma 4: E2B on a Lambda GPU Function, with Google's QAT Weights and Drafter
xbill
xbill
xbill
Follow
for
Google Developer Experts
Nov 19
Serverless Gemma 4: E2B on a Lambda GPU Function, with Google's QAT Weights and Drafter
#
gemma
#
ai
#
vllm
#
serverless
15
 reactions
Comments
Add Comment
10 min read
Moderation
vLLM vs Ollama: which should you use to serve an LLM?
Viswaretas Kotra
Viswaretas Kotra
Viswaretas Kotra
Follow
Nov 19
vLLM vs Ollama: which should you use to serve an LLM?
#
llm
#
vllm
#
ollama
#
ai
Comments
Add Comment
10 min read
Moderation
vLLM 0.34 Stops Reserving 8% of Your GPU: Extensible KV Cache, the Block Size 64 Default, and What to Delete From Your Launch Flags
AI Tech News
AI Tech News
AI Tech News
Follow
Nov 18
vLLM 0.34 Stops Reserving 8% of Your GPU: Extensible KV Cache, the Block Size 64 Default, and What to Delete From Your Launch Flags
#
ai
#
opensource
#
llm
#
vllm
Comments
Add Comment
14 min read
Moderation
How do you host your own LLM behind an OpenAI-style API?
Viswaretas Kotra
Viswaretas Kotra
Viswaretas Kotra
Follow
Nov 18
How do you host your own LLM behind an OpenAI-style API?
#
api
#
llm
#
vllm
#
selfhosted
Comments
1
 comment
9 min read
Moderation
Inside vLLM's Scheduler: Continuous Batching, Chunked Prefill, and Who Waits
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Follow
Nov 17
Inside vLLM's Scheduler: Continuous Batching, Chunked Prefill, and Who Waits
#
vllm
#
llm
#
googlecloud
#
agents
Comments
Add Comment
10 min read
Moderation
Gemma 4 on AWS Lambda with a GPU: E2B on an NVIDIA L4, Day One
xbill
xbill
xbill
Follow
for
AWS Community Builders
Nov 19
Gemma 4 on AWS Lambda with a GPU: E2B on an NVIDIA L4, Day One
#
aws
#
lambda
#
gemma
#
vllm
10
 reactions
Comments
Add Comment
10 min read
Moderation
Tuning vLLM on NVIDIA vGPUs
Florian Lutz
Florian Lutz
Florian Lutz
Follow
Nov 17
Tuning vLLM on NVIDIA vGPUs
#
ai
#
vllm
#
kubernetes
#
nvidia
Comments
Add Comment
8 min read
Moderation
llm-d v1.0 on Kubernetes: KV-Cache Routing, Disaggregation, and Production Readiness
Jangwook Kim
Jangwook Kim
Jangwook Kim
Follow
Nov 17
llm-d v1.0 on Kubernetes: KV-Cache Routing, Disaggregation, and Production Readiness
#
llmd
#
kubernetes
#
vllm
#
inferenceserving
Comments
Add Comment
10 min read
Moderation
Serving 753B MoE Across Nodes: vLLM & RDMA Limits
ram.mehta.1899@gmail.com
ram.mehta.1899@gmail.com
ram.mehta.1899@gmail.com
Follow
Nov 16
Serving 753B MoE Across Nodes: vLLM & RDMA Limits
#
vllm
#
moe
#
rdma
#
dgx
Comments
Add Comment
1 min read
Moderation
Day 5: Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution
Suresh Kumar Pallapothu
Suresh Kumar Pallapothu
Suresh Kumar Pallapothu
Follow
Nov 12
Day 5: Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution
#
ai
#
ollama
#
vllm
#
aiintermediateseries
Comments
Add Comment
3 min read
Moderation
Prefill vs Decode: Why Splitting Them Wins
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Follow
Nov 13
Prefill vs Decode: Why Splitting Them Wins
#
vllm
#
llm
#
googlecloud
#
machinelearning
1
 reaction
Comments
1
 comment
10 min read
Moderation
vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Daniel Sam Pete Thiyagu
Follow
Nov 9
vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade
#
vllm
#
llm
#
googlecloud
#
machinelearning
Comments
Add Comment
9 min read
Moderation
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff
Jangwook Kim
Jangwook Kim
Jangwook Kim
Follow
Nov 10
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff
#
quantization
#
vllm
#
fp8
#
awq
Comments
Add Comment
9 min read
Moderation
Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack
AI Tech News
AI Tech News
AI Tech News
Follow
Nov 9
Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack
#
ai
#
opensource
#
llm
#
vllm
1
 reaction
Comments
1
 comment
15 min read
Moderation
verl GRPO + vLLM Rollouts: GPU Memory Settings for Single-Node Deployment
Jangwook Kim
Jangwook Kim
Jangwook Kim
Follow
Nov 6
verl GRPO + vLLM Rollouts: GPU Memory Settings for Single-Node Deployment
#
rlposttraining
#
grpo
#
vllm
#
gpumemory
Comments
Add Comment
12 min read
Moderation
đź‘‹
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
We're a place where coders share, stay up-to-date and grow their careers.
Log in
Create account