The first time was probably around kindergarden or first grade or when I was about cartoons or comicbooks and I realized most of the other kids (well the boys anyway) were more into football. Which would have been fine accept apparently not giving a **** about football made me the biggest dork ever.
..I still don't give a **** about football.
The second time was later on in school when I found other people who liked comics only the catch was that everybody else was heavy into X-Men, Wolverine, Punisher, etc. and I was the one guy into Superman. We could agree on Spidey though. We also all got into Jademan Kung Fu comics around the same time and some of the early manga to hit the shelves like Heavy Metal Warrior Xenon.
The third time was when I started to feel like I was one of the only superhero fans who didn't think they needed to be all about darkness and deconstruction or "realism". I never really felt a big need for them to be "realistic" neccessarily because the fantasy to me was always part of the friggin point. Also as a related "feeling different" thing I just never really had a problem with good guys being good really or that justice prevailed in stories.
If I had to peg a fourth time it was probably when I got into tokusatsu and I knew there were some of my friends that I would just never, ever be able to convey my reasons for liking it to or even have a snowballs chance in hell of getting them into it. But that's ok because eventually I met plenty of people who did get it.
I don't know it's kinda weird. I think to some degree even among fandoms for things I like I don't exactly fit in wether it be comics, or anime or whatever.