Why am I not surprised that Drive and Gaim are so high on your list? Most of the harshest criticism from Ghost seems to come from fans of those shows. Some of it is to do with petty "rivalry" between shows but generally, Ghost is a break from the last two years' trend of deliberately courting an older audience (that looks set to continue with Ex-Aid) and I suspect a lot of people don't welcome that. No matter how much some may try to deny that they expect Kamen Rider to be for adults, or dress it up with "Good media for children SHOULD appeal to adults!" - yeah, you're not fooling anyone.
I put most of my thoughts in the ep 48 thread so I'll sum them up:
Good: It's very gentle, with some very good messages for children. Love the suits and aesthetic of the show and a lot of the concepts. Also interesting in that it explores spiritual themes that aren't addressed in a lot of other Kamen Rider, and explores death in more depth than any other Heisei series. I much prefer this style of Kamen Rider than one that's trying too hard to be like a J-drama. Akari was a refreshing change from several years of female leads who primarily served as love interests and/or emotional support. Found the actors of Alan and Onari very good.
Bad: Lots of apathy from the production side: notably the long absence of the head writer and cutting its budget to fund Amazons. This, as we see, had a very detrimental effect on the show. Also felt that the more talented members of the cast were often wasted, and many characters were so under-used (Javert, Gyro, Kanon) they probably should have been cut. I don't believe that the show is "unrealistic" or "too optimistic" but have seen some criticism of Takeru saying that all parents love their children (in contrast to Drive's Banno arc); think a more general moral about how family are the people that love us would have been a better one.
I find Fourze a good comparison here - it was written by three people, all of whom had different takes on the characters. Nakashima gave Gentaro slightly more depth, and made Yuuki intelligent as well as perky; she was able to serve as a bridge between Gentaro and Kengo. Sanjo tended to write the characters as more cartoonish and more like caricatures, as well as introducing random subplots that went nowhere since none of the other two writers picked them up (e.g. his attempt to set up a Gentaro/Miu/Shun love triangle.) Hasegawa was somewhere in between. Now imagine if Nakashima had vanished for half the show, left the other two to write the rest with varying levels of consistency between it, and then had to come back and make an attempt to somehow tie it all together. All the while the show is running on a reduced budget because most of the money was channeled into funding a show to pander to an outside demographic. And before anyone comes in with "well, that's a GOOD use of money, Amazons was so much better!!!" - Of course the show that has comparatively more attention and money invested into it is gonna look better.
