I think it has more to do with Labor being an arm of the USSR.
The immigration policy of the UK was instituted by the Labor party. We know who's thumb they were and are still under. It was a way of undermining British society in order to make them more vulnerable to communist influence. A nation with a common identity, history, and societal norms is far more likely to withstand outside attempts to undermine it. In short, the immigration policy was a tool of traitors to bring down their country from within.