Well, probably a few things, but the one that jumps immediately to mind is that you're not controlling his arms.
On topic, I seem to have huge problems with the butterfly guard. Every time I try to use it, my opponent either controls my legs and passes, or just stays and is able to avoid being swept. I really can't figure out what I'm doing wrong.
Make sure that you have a good grip (usually an underhook and an overhook, if you're going for a straight butterfly sweep). If he's getting control of the leg it means that you don't have enough control with the hooks.
Also make sure the butterfly hooks themselves are deep. Keep your knees in tight to his ribs and keep the hooks in his groin until you work for the sweep, then slide the feet out to under his knees to control him and go for the sweep.
When people have problems with the butterfly guard, it's not a matter of tricks (though good transitions to the half guard and to the X-Guard are nice to hard). It's almost always a matter of screwing up some of the details.
Do you have a video or pics of how you're doing it? Because if the advice there doesn't work I'll probably need to take a closer look to be useful.