h200 with wide-ep

(blog.vllm.ai)

45 points | by robertnishihara 11 hours ago

2 comments

kingstnap 2 minutes ago
Impressive performance work. It's interesting that you still see these 30+% perf gains like this.
Makes you think that you will continue to see costs for a fixed level of "intelligence" dropping.
danielhanchen 0 minutes ago
Love vLLM!