Rogers is no joke, but in a rematch he'd get Fedor'd twice as bad. And a hard loss twice in a row is very damaging for your career.
Right now, if Rogers wins two more fights convincingly against top 10-15 competition he's on his way.
But if he rematches Fedor and gets KTFO or submitted first round, he'll have to beat maybe five or more people to get back on track. Career wise it just doesn't make sense.
Rogers should be calling out Werdum, a guy who stylistically he is a very good match with. Plus then he has two solid wins over top 10 competition.