You know what, I could care less if (hypothetically) Bryan Singer adapted the next Harry Potter book into a sex-crazed, drug induced, rated R picture....if someone asked him to direct X-men 4, I'd sure as hell want him to do it knowing how fantastic X-men and X2 are.
In Raimi's case, he's given us 2 good films out of 3. And I wouldn't even consider 3 his film due to the shoving of Venom down his throat by Arad...if it was his vision of how he originally wanted the film, I'm pretty confident it would have been just as good. He more than deserves a chance at this given his track record.