This is just a guess from what you have written. If there was a shorted output this could induce noise into the amplifier or souce unit, which ever is connected directly to the speaker. And when the short is temporarily separated the noise will stop and the speaker will play. Or the buzz could be the speaker intermittently and rapidly shutting on/off as a poor tinsel lead to terminal or tinsel lead to voice coil connection is misbehaving.
Gently push in on the center of the speaker cone while it is playing and see if it behaves differently while playing or buzzing.
Inspect the wiring, both speaker and other, directly behind the speaker.
Swap out the speaker, jumper a new speaker wire, jumper a new RCA or reverse the channels of the RCA, first at the source end and next at the amplifier end. Reverse speakers, speaker wiring, amplifier channels one at a time. This sequence will reveal what part of the signal path or which component is the problem.
All it would take is a single wire strand from a frayed connection to create this issue. So physically move or press every related connection and harness that you can access while having the problem.
It wouldn't hurt to have an inexpensive multimeter to help with the diagnosis. If you learn how to use one, I promise this will not be the last time it will be needed.