It all depends on the board, your skill level, and your preference. Some boards may ride better without the fins if you're advanced enough to handle it.
A lot of people/pros take the fins off when they really want to make themselves dial in a move. Having less traction in the water makes you do everything exactly right to be able to stand up the move and ride away. Then, the next time you do the move with fins, it's a piece of cake.
I just got the new Temet. I would never take the fins off because it rides so loose in the water. Like I said, it depends on the board, rider, and skill level. It's all personal.
www.enemyofevilclothing.com