entia non sunt multiplicanda praeter necessitatem
For those of you who don't read Latin, that translates as "entities should not be multiplied beyond necessity," and it's one of the most important concepts of logical thinking - Occam's Razor.
Occam's Razor (also known as Ockham's Razor, since it was named after William of Ockham, the 14th century Franciscan monk who first postulated it) is more commonly phrased as, "all other things being equal, the simplest answer tends to be the right one." This is usually applied when multiple theories are used to describe a situation; the theory that makes the fewest assumptions and the fewest entities tends to be the most accurate theory.
One of the main reasons we prefer simpler theories (according to philosopher Sir Karl Popper) is because simpler theories apply more broadly than complex theories, and thus they are more easily tested (and refuted). Since valid scientific theories can never be proven, only disproven, a theory that can be more easily tested and refuted is preferable to one that cannot be tested and refuted (in fact, theories that cannot be disproven are not valid scientific theories).
Occam's Razor is not a scientific theory. Rather, it is a heuristic method used for choosing among competing theories. While there is some risk of eliminating valid theories, probability theory and statictics argue in favor of Occam's Razor on the basis that all assumptions introduce possibilities for error. Thus, theories with more assumptions are more likely to be incorrect. Additionally, simpler theories will be easier to test and refute, bringing us back to the more complex theories that we originally bypassed.
If you have trouble remembering Latin (like I do), you can just keep in mind the KISS principle: Keep It Simple, Stupid.
