Orbit · Explore

AI Lab · by Rohan Mehta · 2026-10-17

Fine-tuning a small model on your own docs: worth it?

Tried LoRA on 5k internal docs. Tone improved, factual accuracy didn't. Retrieval did more for us. Anyone had a different result?

27 sparks · 19 comments

Comments

Mariam Said · 12 likes

My manager read one article on fine-tuning and now it's our whole roadmap lol

Noah Meier · 7 likes

Good point, didn't think of that

Omar Al Falasi · 9 likes

Not gonna lie fine-tuning is one of those things that sounds simple until you try it with real constraints :)

Tiny Noodle · 6 likes

I'm more of an enthusiast than a pro but fine-tuning has been fun to tinker with on weekends!

Finn Brown · 4 likes

send help

Rohan Mehta · 4 likes

The beginner friendly path for fine-tuning is way better than a year ago, that part is underrated.

Budi Santoso · 3 likes

lol my cat has stronger opinions on it than I do

Maya Williams · 3 likes

We tried fine-tuning last quarter, worked great for small stuff, got messy at scale

Mia Anderson · 0 likes

we saw the opposite honestly

Yui Tanaka · 2 likes

Would love to see actual data on fine-tuning instead of vibes. Anyone have links

Mateo Cruz · 6 likes

Fair enough

Maksym Bondar · 1 likes

Is anyone else tired of the hype around fine-tuning, feels like every other post is about it

Mia Anderson · 2 likes

fair, but I'd still try it once

Curious Baker · 0 likes

Same experience here

Camila Rocha · 0 likes

As someone who does this for a living, the biggest cost with fine-tuning is the stuff nobody puts in the demo

Midnight River · 12 likes

we saw the opposite honestly

Curious Mango · 4 likes

can you share what you changed

Tanvir Rahman · 0 likes

cost vs benefit matters here, we ran numbers on fine-tuning and the payoff came later than expected

Maia Brown · 0 likes

we saw the opposite honestly