Is racism always necessarily wrong?

Thinking about it, every other negative thing someone wants to say about someone else... however cruel or nasty... won't be shut up because they claim it impinges on their 'freedom of speech'. When it comes to an opinion on a race of people, you're not allowed to say a word!! There is something wrong there!!