Orbit · Explore

AI Lab · by Cedar Coder · 2026-10-26

Fixing tool-call accuracy in vLLM: great writeup on why your open model 'can't use tools'

ubicloud.com

Ubicloud dug into why tool calling with open models served via vLLM was underperforming, and a lot of it comes down to parsing and templating issues, not the model itself. If you've ever blamed the model for a broken JSON tool call, worth a read.

20 sparks · 3 comments

Comments

Sophie Gagnon · 10 likes

this is why hosted APIs still feel better even when the open weights are comparable, all the glue is done for you

Mariana Costa · 2 likes

yep, the model is 60% of it and the serving stack is the other 60%

Maya Putri · 2 likes

chat templates are the silent killer. spent a week debugging what turned out to be one missing token in the template