Do Azura's Star quest.
The reward is the Azura's Star, a re-usable grand soul gem, it never runs out, can use it over and over again.
Also, exploring is a big part of the series, do quests until you don't have any that interest you at that particular time, then go out and explore/find stuff. I love this game.
"You lack the requisite spine and testicular fortitude to study under me"