How do I tell when I don't know when I think that I do?

Trying to use logic that you do not understand can lead to fallacious logic.
I obviously want to avoid using flawed logic, meaning that I stay away from topics that I don't understand.
The problem with that ignorance on certain topic can be disguised as knowledge.

I remember this happening to be once a few months ago with the modal logic axiom s5 when I thought that it was "It exists in some possible worlds = exists in all possible worlds" when it was more complex than that. The reason why I made a argument against the axiom is that the modal argument for god third premise required it and the argument would fail without it . The (flawed) argument that I produced was attempting to show that axiom S5 is incoherent if one reject the analytic-synthetic distinction (The modal argument for god does not work anyway if one does accept the analytic-synthetic distinction anyway).
I presented the argument to a friend he (rightly) rejected, here's the argument I produced below.

1. Axiom s5 (X in some worlds = X in all worlds) is true.
2. It is possible that a unmovable objects exists.
3. If it is possible that a unmovable objects exists, then it exists in some possible worlds. (from 2)
4. If a unmovable object exists in some possible worlds, then it exists in all possible worlds. (from 1 and 3)
5. if a unmovable object exists in all possible worlds, then it exists in the real world. (from 4)
6. It is possible that a irresistible force exists.
7. If it is possible that a irresistible force exists, then it exists in some possible worlds. (from 6)
8. If a irresistible force exists in some worlds, then it exists in all possible worlds. (from 1 and 7)
9. If a irresistible force exists in all possible worlds, then it exists in the real world. (from 8)
10. The idea of a unmovable object and irresistible force existing in the same world is self-contradictory. (from 5 and 9)
11. all self-contradictory ideas are false.
12. any axiom that creates self-contradictory ideas are incoherent. (from 11)
13. Axiom S5 created this self-contradictory idea. (from 10 and 11).
Conclusion: Premise 1 cannot be true. (from 12 and 13)

The problem with this is that it is a straw man argument and that irresisability and immovability can be Localized. (Omnipresent is an example of a trait that cannot be localized)

First let me address your general question, which if I understand it correctly, is the problem of thinking we know something we actually don't know, or to put another way, simply being wrong about something. There is no sure-fire to avoid this, but you can use hypothesis testing to help see if you are wrong. With this strategy, you try to prove yourself wrong (not right). You look for DISconfirming evidence. If you cannot see how you can be wrong, then wait for other people to point out how you might be wrong, and consider their reasoning carefully, rather than dismiss it because of the desire to be right. In short, put truth above self-validation.

As for your specific example, this is above my pay grade :) A little too philosophy heavy for me, so perhaps someone else can shed some light on that part. Off the bat, I have a problem with the premise that "X in some worlds = X in all worlds," which seems to me like a simple assertion that can't possibly be supported empirically, and would require some linguistic trickery to "establish."

I am way beyond that of a novice, I'm a step below the lowest form of novice. But I agree with Bo on the first statement if it doesn't refer to God as being represented by X. If its meant to then it isn't logically correct to assert that an immovable object is analogous to God or that any subject matter can be inserted into the X as the subject. When you do inject any subject into X you are claiming anything can have omniscience. I may be completely wrong as to what the argument is trying achieve, but the modal ontological argument for God says the opposite of your opener. Its says If a God exists then that God must exist in all possible worlds which also maintains the some worlds as being true too. But that premise or opener wouldn't apply to all subjects. But like I said, I'm a prepubescent novice of novices so I might not even be coherent when I say what I think the argument means.

Not omniscience , omnipresent if you insert anything into X, which still wouldn't work.

Didn't mean you're claiming x is omniscient , meant omnipresence