There's definitely a form of moral panic over immigration that isn't really justified. Everyone enjoys a good moral panic though and it's certainly going to sell more newspapers than printing gigantic tax reports on the front page. I'd prefer to see the state changed so that regardless of numbers we couldn't start making stupid decisions simply due to popular backing. Of course that won't get the necessary popular backing to happen
In reality we'll probably make some silly, counter-productive decisions and end up putting immigrants through the sort of complete shit that'll lead to their children becoming more aggressive towards the state they end up living in because it didn't offer their parents a single iota of respect. Maybe then we can stick them into concentration camps or something, I hear that's worked out well before.
I'd like to point out as regards dda's post that, ignoring whether or not there really is an acceptable rate of immigration in terms of overall population size, there's no reason this can't reach a really high number due to the influence of other factors.