‘Pitting so-called local workers against temporary foreign workers is a red herring,’ says migrant advocate