I agree with you. The context is ‘whom’ he is trying to persuade…in this case, an entirely male ‘opposition’ who stood steadfast against renewing the Act for reasons having more to do with biases against immigrants, gays, and native women, albeit many also have biases against women and their freedoms/civil liberties.