Would 15 to 20 pound dumbbells be a problem then?
I've seen many boxer's do that, but mostly in short "sprint" type drills. Further, I've only seen them done with uppercuts. The problem with too much weight in the other types of punches is that you run the risk of injuring yourself by pulling muscles or stretching ligaments in your joints.
If you are simply doing shadow boxing for extended periods of time (multiple 2 or 3 minute rounds) stick with just 2 or 3lb weights. Further, you want to concentrate on form and keeping you punches short and tight. Don't put any follow through on your punches when your shadow boxing with weight.
It doesn't matter if you agree or disagree
... As long as I don't bore you and I spark a moment of thought, my goal is achieved
Queng leon queng tigre ecu tacacut, queca pa? - Pampangan Mandarigma Motto