The word itself, on its own, in no other context, can be considered as incitement to racial hatred, but in addition to this, the heinous context in which it was used (as part of a well-known racist rhyme, whilst knowingly being filmed for a national TV show) makes it uncontestably so.
What would you say if your child's schoolteacher used this rhyme and got your child's class to recite it? Would you be defending that person too? I'm very surprised that someone as astute as you is questioning this. You ask what should his punishment be?... whatever the law of our country decides, if he were to be found guilty. Personally I think in this particular situation a conviction would be punishment enough, but I am not the judge.
The " I tried not to say it" excuse is pitifully similar to the sort of thing a seven year old would say to his teacher, but less excusable, because he isn't a seven year old, he just behaves like one.
I cannot fathom why you are trying to defend a man who has admitted that he was totally wrong - Even he is not on your side.
...and if you want to provoke me into answering the question in your first sentence - clearly the answer is yes - and you seem to be quite comfortable there. I feel sad that you are happy to parade this disgusting word as you do. Why not apologise, as Mr. Clarkson quite rightly has already done?
Get a life man