It seems like every time I ride I am straining my neck pretty good. Its always both sides of the back of my neck the two muscles or whatever that run from the back of your head down your neck.
The only thing I can think of is that when I catch an edge and slap the water good with the back of my head I am doing something. I usually have at least one of those when I ride and ride about three times a week.
It's almost certainly your hard falls that are causing the problems. The back-of-the-head plants are brutal on neck muscles. If you have three of those per week, your neck is definitely going to be sore a lot.
Mine kill me after every time out. If you don't want them to hurt, just cruise around. If you want to progress, you're going to hurt, just part of the sport/love/addiction.