Ever swapped speakers within tower speakers?

I have a pair of Fusion tower speakers (Diamond Audio) that look like this:



I like the look of the enclosure, but I don't particularly care for the sound quality... No mid's! The speakers are 6.5" and 5.25"... I'm thinking I can swap them out with a better quality speaker instead of buying new tower speakers.

Has anyone ever done this before? Any tips or warnings?

I'm looking to replace them with these:
Alpine - Type R 6.5" 2-way coaxial
Alpine - Type S 5.25" 2-way coaxial

The small tweeters on my tower speakers don't work, and I'm not interested in replacing them. I guess they'll be there for looks only!