He... wasn't really "racist." At all, in fact. The guy was a legitimately different species, one that had the ultimate end goal of wiping out all other species on the planet so they could win their game and so theirs could rule. While Hajime himself may not have been of that mindset, it's not exactly racism if he's just the exception to the rule - and the guy did get over the issues he had with Hajime eventually.
It's not like "if every [adjective] person you've ever met is like this, that doesn't mean they all are" situation - they all WERE that thing, that thing being, in this case, monsters bent on ruling over all other species. It was in Hajime's blood and that's why the ending was what it was. He would have had to fight until the end had Kenzaki not done what he had, because despite his feelings, it was still who and what he was. Kotaro's (was that his name?) wariness was perfectly understandable and it was really just self-preservation - a form of survival, which was one of the show's running themes.
If Hajime had been, say, like Kenzaki the whole time, it would have been one thing, but keep in mind he didn't exactly act nice about even his unquestionably good actions. It makes it easy for someone who doubts to question his motives, especially when family is concerned, when Hajime was only ever nice to the little kid and was otherwise, for the most part, pretty standoffish and even antagonistic at times.