That sounds quite rude (in various ways), but it is really intended as a reference to a story from history - one which it is hard to escape from. By now we ought to have dropped the joke, as even in biblical times the sins of the fathers only reached the third or fourth generation. But the story is too good to be ignored.
One dark and stormy night during the Napoleonic Wars, when the whole of England had been primed to expect a French invasion, a French ship was blown ashore at Hartlepool. Only one survivor reached the safety of the land alive, dressed in a French naval uniform. That poor survivor was not human - it was probably the captain's pet monkey dressed for the amusement of the crew.
Of course it could not speak English. The fact that it could not speak French either was hardly relevant. How would anyone know what French sounded like anyway?
The people had never met a Frenchman but it was obvious that this was no Englishman. Indeed they had probably never even heard of monkeys.
Inevitably, for the safety of the realm, the poor creature was hanged as a spy and the story has gone down in legend.
It seems that the people of Hartlepool (in general, at least) are not offended by the terminology. Perhaps they have grown up with it and developed broad shoulders. Even the official mascot of Hartlepool Football (soccer!) Club has the name 'H'Angus the Monkey'.
For the rest of us it is a rather nice tradition to tease them gently. It was a perfectly understandable error, under the threat of invasion by a strong foreign power.