Mostly depends on the time of day you're shooting. Light conditions will greatly affect your color. Early AM and evening will net you the best light, but in the summer, between 10AM and 3-4PM I wouldn't hold your breath for good color.
The beginning of this video was about 12 PM, and we continued filming til sunset. You can watch the color quality become distinctly better as the day goes on. This was shot on an HD Hero 2 (a few random shots with the handycam though). Some of the color sort of pops in the midday shots, but I have had a huge issue in past videos where the GoPro Hero one footy came out real washed out like you're saying. Glare from water and stuff and possibly fog will affect it too.
from Nick Schrein on Vimeo.
The environment also plays a big part. Under trees during that time? Less harsh light to blow out the footage and you'll get a better color quality, but go out in a field and you'll overexpose it. I think there is some setting on there that you can change, but I haven't messed with most of those since we usually shot video in the mornings and evenings.
EDIT: Sorry, I missed the bit about conditions. You should check out the exposure settings on the camera. Like I said, I haven't messed with those, but they may help you out.