Actually you can. Every normal human being can fight naturally. Usually martial artist (mostly those pseudo traditional ones) suggest that without martial arts training the fighting ability would be at 0% and that there would be a clear line between a martial artist and a non-martial artist. Try to take away the newborn baby of a mother and she will prove them wrong. Sure, specific fighting training can (drastically) improve the fighting ability though and that's were I am with you again. Sparring is an absolute important factor to get as much access as possible from your maximal natural fighting potential. So for (successful) competition you imperativly need sparring.
Okay, I see what you mean. I guess I could rephrase and say "if you don't spar, you can't fight well
." (It should also be noted that being able to spar is by no means a gaurantee that one can fight well, but it's definitely a prerequisite.)
And yes, it's true that some people are just tough and can scrap, but VERY rarely can they match up to somebody who is well trained. There are exceptions to every rule, of course, but the rule remains.
Overall, I agree with your sentiment.