As much shit as Scarborough gets, it's really pretty decent. Orton Park (a small part of the Scarb) is bad for crime, and that's where Scarborough gets most of it's bad name. It's definitely not on my shit list like Jane/Finch, Regent Park, Rexdale, etc. Toronto as a whole is a safe place to live (by the numbers), just a lot of petty crimes. Pickering gets a lot of the wannabe gangster run-off from Scarborough, which to me is just as bad, if not worse.
The funny thing is that Pickering and Scarborough have two of the nicest neighbourhoods I've been privy to know people in, which are Rougemount and the Bluffs. In general, I'd consider both Pickering and Scarborough in general to be middle to upper-middle class areas.
I live in Port Union, so thats close to Dean Park and Generation. I went to school in the Guildwood, Bluffs area, lots of pretentious kids. And now I work in Pickering, lots of wanksters here.