I agree with most people here. It was more than likely a compliment. I don't think that he was implying anything negative by saying it either. He is bigger so he probably thought that he would just out-muscle you and was surprised when he came to the realization that he couldn't.
Sig Credit: M.C