Ad hominem

Is there a specific name for an ad hominem fallacy where someone targets traits that they don't even know that the other person has? For instance, I've seen it happen where they make an assumption about the size of the other person's certain anatomical parts! It's just that it's different to making comments about someone's weight or hairstyle or something, because at least they're traits you can see, so I'd have thought there was a case for recognising it as a specific category

There are other possibilities that are not classically fallacious or necessarily ad hominem abusive such as amazing familiarity, loaded question, or just plain insult and ridicule which might be more of a non sequitur. 

For instance, if someone claimed that women prefer men who are physically less endowed, and the response was well, you should know. There’s no real logical fallacy it’s just a sarcastic retort. 

It is ad hominem because it derails the conversation from the topic and directs it “to the man”.

The whole point of logic is to develop techniques for evaluating the cogency of arguments independently of the arguer's identity or in this case their physical characteristics.

The main distinction is whether the person being criticized is arguing or testifying?

Are reasons being presented, or must we take the person's word for something?

If the person is arguing, the argument should be evaluated on its own merits; if testifying, then credibility is important.”

Whether it may or may not actually be fallacious is not relevant.

For instance, if the person testifying about female preferences for the size of men’s penises is a famous transgender author then their credibility might be a valid ad hominem argument without being fallacious.

The OP made me chuckle, heh. 

I imagine (and I believe you're imagining) something like this:

Dominic: "Just because X happened, doesn't mean Y is true."

Antonio: "Yeah, I don't think we can trust the words of a guy with a one-inch pole."

This style of response, while somewhat amusing, is fallacious (usually just ad hominem (abusive)). There's no specific name for when the part of the person being attacked is seen or unseen (logically, insulting someone by weight, or manhood size, are the same thing).

Another could be:

Violet: "I wish people would realise A and B are not mutually exclusive."

Kathryn: "You're saying that because you can't get laid."

This sounds more like ad hominem (circumstantial), but it's fallacious for similar reasons.