Sitting for long periods is why you, me, and lots of other people have tight hip flexors. Any muscle that rests in a shortened position for long periods of time will naturally shorten (the opposite of stretching). Also, if you do a lot of situps for exercise, keep in mind that situps are half hip flexor, and half abs. Workouts like P90X Ab Ripper X will tighten up your hip flexors, big time.
Lunges are a glute exercise, so while tightening glutes along with your flexors brings balance to the pelvis, it's not going to stretch your HFs.
The stretch with one knee on the ground (kneeling) is the best way to isolate the HF for a stretch. The key is to not lean forward, but sit straight up like you're lunging, and push your pelvis forward. You'll have to engage the glutes (butt) to push the pelvis forward for this stretch, otherwise your tight HFs will pull on the pelvis and it will tilt, and you might miss the stretch all together. So, tight butt for that stretch (pelvis tucked). You can also take the foot that is behind you during that stretch and place a basket ball underneath it. So now the foot is elevated, the knees is in full flexion, which stretches the quad and HF.
Rolling out the hip flexor on a medicine ball also works great. It's going to hurt, but it works.
Mine hurt after I stretch too. Stay on top of it and be consistent and that will work out.
Stretching should be a corrective exercise used to loosen tight muscles that prevent you from moving correctly. Stretching muscle that are not tight before you ride is not a good idea. So you're right. Don't just stretch for the hell of it. Just stretch out the imbalances. Over stretched muscles can become under active.