Here are the most basics of the basics of system tuning.
Crossovers: Starting with the highpass section. It takes four times the excursion for a speaker to reproduce one octave lower at the same amplitude. So obviously at that rate you will eventually hit the wall in both the speaker and amplifier abilities. Limit your tower speakers and in-boat coaxials to a bandwidth where they can reproduce the music cleanly. This will produce more output. The lowpass subs should be symmetrical in frequency and output at the crossover intersection with the highpass in-boat coaxials in order to sound musical.
Gains: You want unity gain at every point and component in the signal path. That means that every component beginning with the source unit and extending to the amplifier should reach their maximum clean potential simultaniously. This will produce a lower noise floor and the highest dynamic range. Your system will also operate more efficiently. And you may limit your amplifier gain to protect your speakers from distortion.
When setting up your system you should have good voltage, well-recorded & dynamic program material that covers the entire music range and that is somewhat busy (a CD is best but if its an MP3 file make sure its a clean download with no equalization), (it may not be your favorite music to be ideal for system set-up), all tone controls, EQ and Bass Boost flat.
Our basic system set-up is a multi-page draft plus an advanced version. Its not something that I can get deep into here. But these are the basic principles.