when opponent asks to ignore certain information

I have been trying to figure out if the following statement is a fallacy for a while. It usually crops up when I'm having a more philosophical debate with someone, so I'm not sure if it can be logically fallacious, but it feels very wrong to me.

Side A : According to source X your viewpoint is wrong.

Side B: Let's not bring source X into this, because then the whole argument is null.

I understand this to mean that if we both agree that source X is valid, then there would be no point in debating this issue.

Is this fallacious? When can this be a valid argument? How can a debate move on from this statement to a productive conclusion?

"According to source X, your viewpoint is wrong" - we need to flesh some things out here.

Did the person make a factual claim that source X disproves? (1)

Or did they give an opinion that the author of source X disagrees with? (2)

In the case of (1), this is a fair maneuver on side A's part, here's an example:

Alice: The Moon landings took place in 1968.

Bob: Actually, it happened in 1969, as source X shows. 

Side B ignoring evidence is not necessarily a fallacy as much as it is simple ignorance, as it depends on why source X was ignored. If it was ignored because of its origin (e.g. a Trump supporter dismissing a fact-check from Snopes, because it is "liberal") then this is a genetic fallacy.

In the case of (2), simply pulling out a source that disagrees does not prove the opinion is poorly-supported, wrong, or anything else (this is borderline appeal to authority/appeal to false authority). You would need to explain source X and why you think it supports your point.

Ah, the battle of the experts. This is a discussion of investigation.

The place to start is more fundamental than what X says or does not say. What is critical is the basis for what X claims. That requires disregarding X and doing your own investigation. 

A proper investigation should be transparent, objective, data-driven, inclusive of broad expertise, subject to independent oversight, and responsibly managed to minimize the impact of bias or conflict of interest.

I think this would be an example of cherry picking 

They are ignoring evidence that is inconvenient for their position in order to focus strictly on the evidence that might support it

As for whether this argument could ever be valid, the way it's presented in the OP, the argument is not deductive so philosophically speaking this argument can't be judged in terms of validity. It is possible to word this argument in a way that would be logically valid:

"If we consider the evidence (E) then the debate were having is meaningless. If we want to have a meaningful debate we must not consider E." 

This would be a logically valid way to put the argument. However, it would still be fallacious because the arguer is attempting to ignore evidence that does not support their position

maybe genetic fallacy or some kind of Biases. I would depend on the reason for doing so.