There's really no way to lower the rudder as it uses a cotter pin through a non-adjustable hole to hold it in place. I also have a 2004 Supra and had to replace the rudder a couple years ago when I hit something and knocked the rudder clean off. I went through
www.skidim.com for the new rudder as they were a good $100 cheaper than getting it from the dealer. Ski-dim had Supra drop ship it straight to my door, so there is no worry of getting non-OEM parts from them. I want to say that the rudder was around $250 ish?
It's actually very simple to replace, though it's a bitch to get to if you have a V-drive. You'll need a buddy to hold the rudder up through the hull while you slip in the cotter pin. Just make sure you thoroughly lube it up with some marine grease before you install it. Then re-attach the steering linkage and you're done. That;'s pretty much it, so unless you have some hull damage there's really no reason to pay your dealer $400-$500 to do it for you.