In the complete context his comments seem to be a way of explaining why he's not more popular with a certain demographic.
That's what it seems like to me. I would expect Clinton to say the same thing (or similar) about why she's not as popular among African Americans without making it strictly a race issue.