Not translating silence

I've been noticing how much effort I used to put into making silence mean something. Now I'm trying to just let it be there, like a rest in music that doesn't need to be filled. It's harder than it sounds.