What do you lose when you cut the silence?

I was editing a scene yesterday where a woman stands at a window, watching rain. The original had ten seconds of quiet before she spoke. I cut it to three. The director said it felt 'tighter.' But now I keep wondering—what did we lose? Not just time, but the weight of waiting. The way a city breathes between sounds. I don’t know if it’s worth saving, but I miss it.