I never have to.

I sleep on top of my comforter and use my soft blanket as a cover.