A big reason why casting sometimes bombs the way it does is because of scheduling conflicts. Sometimes they have prior engagements which makes it difficult for them to come back to shows they were once on.
Also, sometimes management gets in the way or there's just the plain ol' lack of desire to comeback and reprise their role. Especially if the role was a major handicap in their career later on.
I'll always remember what Takeru said in an interview, in the acting world playing in shows like Kamen Rider or Super Sentai was always considered bad luck. Once you played that role you never recover your career.
For some reason that's change lately, for him and for Yuusuke Yamamoto (Tsurugi), Yuuki Satou (Kagami), and Hiro Mizushima (Tendou).
But for many it didn't turn out that way. Though, Shinji's actor had a pretty steady career... GaoRed too.