I was thinking my board was too heavy too, but when I tried a smaller board, I found I just couldn't get enough speed to get any pop and the landings on the smaller board were anything but pretty.. felt like the board was "sinking" too much.
The ladies do seem to have more natural finesse but try to steer the board more than lift it and resist the temptation to get too small a board. Often guys will learn the hard way and try to get more air by jumping harder and lifting more.
Look for a local shop that will let you try different boards and try and get rides on other boats and try out different boards. If you have a good set of bindings and are friendly about it, I don't think people will mind switching bindings once or twice on the boat, as long as it you're not holding up on-the-water time too much. Use common sense, switch out between your sessions, they're not going to want to wait while you switch boards after every fall.