Question: People are saying that if you're going to let immigrants in you must be willing to open your house to them.

I guess that's a non-sequitur. Maybe a response to that would be "If you let certain people move in to your neighborhood, does that entail that you have to accomodate some of them in your house?", and since this obviously is ridiculous and the answer would be no, follow up with "Alright then how does letting certain people inside the borders of the entire country necessitate that you let some of them inside your house?"