Its always on the you to decide honestly. I give my thoughts on what i have trained.
BJJ is hard because you will have to deal with being dominated a lot, its not as hard in that respect as wrestling though, it might be the easiest to get to grips with though, really hard to master.
Kickboxing is hard to learn technically, although maybe not as much for a Karate guy...most spend a long time looking and feeling like a retard before the breakthrough comes and you start kicking right.
Muay Thai is the toughest, you have to be damn tough to keep coming back to Muay Thai - just clinch training will drain you into nothing and leg kicks are the worse.