With all due respect, I don't think anyone was suggesting such a thing. What I've been told is that once you have 2-3 letters from your specialty, you should supplement with attendings from related specialties. For example, if you're applying to internal medicine, getting a letter from someone in neurology advocating your clinical skills and why you'd be a good IM physician can help. And vice versa.
If you're applying to peds, a letter from an EM attending who can speak about your clinical skills with kids (if you're in an ER that sees peds patients) is still related. If you're applying to family medicine, a letter from OB-Gyn would still be relevant (as would a letter from IM and peds).
That's just what I (and apparently the OP) have been told. Obviously, surgery is different since you'll need letters that address your skills in the OR, but for a lot of other specialties, there's overlap, especially when talking about overall clinical skills, interaction with patients, medical knowledge, etc.