35 and torn between bigger home now vs smaller home + bigger portfolio later

Age 35, one kid, maybe a second. We're comfortable but not fancy. We could stretch for a bigger place now or stay put longer and keep investing aggressively.

Current portfolio:
- 70% equity funds
- 10% bonds
- 20% cash

Goal is both family stability and long-term freedom. Risk tolerance medium. This isn't purely a math question, I know, but I'm curious how people framed the tradeoff without turning every dinner into a spreadsheet fight.