When you remember a book years later, what comes back first?

I'm shelving returns tonight and it hit me that I almost never remember plot. I remember where I was, what the light looked like, how the spine felt. Do other people actually remember the story, or is it always the furniture around the reading that sticks?