Also, understand that when you attack, you automatically open up your defences. If your opponent manages to slip a punch and nail you in the body, chances are, if you're holding your breath his blow will knock the wind out of you and we all know that's not cool. This risk is however eliminated when you exhale quickly during your own attack.
And, like Liddellianenko said, you will be able to exert harder while exhaling. This goes for punches too. You'll use less energy throwing a hard right straight while exhaling than while holding your breath or breathing in. You may also find your punches, especially your jabs, will be much quicker and stronger, provided this technique is accompanied with proper form of course.