It's a bit difficult to respond to this since you seem to be switching your stance. In your first few posts, it appeared that you were criticizing the existence of an external world on philosophical grounds- perhaps the idea of "physical" is meaningless, etc. Now, it appears you are criticizing the external world on more practical grounds- simulation argument, etc. The fact is, if we are in a simulation, then that simulation is at least in a "real physical world". While it is logically possible that we are in a simulation, there is no reason to assume we are until we have proof.
Okay, if we are in a simulation then that simulation is in a "physical world". In this situation, a physical world is assumed to exist. If a physical world is assumed to exist, we do not assume we are in a simulation for the same reason we do not assume that God exists- lack of proof. If there is no external world, then we cannot be in a simulation because nothing outisde of our consciousness exists. If the physical world does exist, it should not assumed that we are in a simulation until proof has been attained.