I really hate the brainless trope “all models are wrong, some are useful”.
Any model only approximates reality. A model is more useful to the extent it approximates better than an alternative.
A better saying would be: “all models are wrong, but some are less wrong.”
@tiago Or: "all models are wrong and useful to varying degrees, select the one that fits your needs for accuracy and computational efficiency". Not as pithy, but arguably the most accurate.
@johnabs I don't think this extreme level of subjectivity is a meaningful representation of the scientific method. We don't really get to decide which model most appropriately matches reality. We need to check.
Useful to read in detail what Box himself meant with the quote: https://en.wikipedia.org/wiki/All_models_are_wrong
It's not about the scientific method; it's about the utility of models *in context*. Models can be used for more than just scientific inquiry, are ARE used for non-scientific purposes more often than the alternative. That is to say, the utility of any given model is not a myopic, one-size-fits-all definition that is universally applicable, but is, in fact, defined by the person(s) who are using it. Hence, "this level of extreme subjectivity" is unavoidable, unless you happen to have a universally applicable definition of utility.
He even says in a quote on the wikipedia page: "All models are approximations. Assumptions, whether implied or clearly stated, are never exactly true. All models are wrong, but some models are useful. So the question you need to ask is not 'Is the model true?' (it never is) but 'Is the model good enough for this particular application?'"
For example, the epicycle model of the geocentric universe was completely wrong, but it was useful for the people who developed it, in that its purpose was to predict (with a surprising degree of accuracy) the behavior of visible planets. Just because it was extremely wrong in terms of being an accurate representation of reality, didn't mean it wasn't useful for the people that developed it and their purposes.
If you use a linear regression to match a logit/probit curve, who cares as long as you know you're going to stay in the semi-linear regime? (this is done all the time in instrument calibration, for example) If you want to reduce your logistics and shipping costs, a TSP approximation is better than waiting 1e9 years for THE answer. So I can still pick and choose the level of utility I need for my application (whether it be absolute truth, or simply an improvement, or a "good enough" approximation), and, as I said in my original comment, you may select the model that suits your needs.
I did read it, the entire quotations section, and even included a quote from that section (from a book Box authored himself) that contradicts what you're saying. He literally asks "is the model sufficient for our application?". Thus, it is the application that defines the utility, and more applications of models exist than solely the scientific one.
And, clearly, we're not talking about models being right or wrong. It's already assumed they **are all wrong** (and factually, they are), we're talking about the utility of models which are assumed to be wrong a priori, and your *personal beliefs* about what constitutes utility is not universal.
And I find it ironic that you think I'm projecting when you didn't deal with the quote I presented directly from Box's book. Have you considered the possibility that your interpretation of his words may be the incorrect one?
Let's go through some quotations from Box:
“Since all models are wrong the *scientist* cannot obtain a "correct" one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so overelaboration and overparameterization is often the mark of mediocrity.”
“For such a model there is no need to ask the question "Is the model true?". If "truth" is to be the "whole truth" the answer must be "No". The only question of interest is "Is the model illuminating and useful?".”
”... all models are approximations. Essentially, all models are wrong, but some are useful. However, the approximate nature of the model must always be borne in mind....”
It's obvious that the context in question is a scientific one, where one tries to approximate reality.
You show a deep confusion between subjectivity and *arbitrariness*.
Personal beliefs are arbitrary. A person can believe that the moon is made of cheese, and that they are napoleon, etc.
The utility of a model is not arbitrary. Although it is not universal — it depends on the situtation — they are not *arbitrary*.
And, independent of context, the utility of a model of reality is primarily *ALWAYS* connected to how well it approximates it. This is never an irrelevant aspect. To reject this is to reject science.
Disregarding your internet psychoanalysis of my "deep confusion" on something that didn't even come up in the conversation, it seems that you actually agree with me:
> Personal beliefs are arbitrary. A person can believe that the moon is made of cheese, and that they are napoleon, etc.
To clarify, the personal belief I was referring to was your equating a model's accuracy with its utility. That IS a personal belief, and is the belief we disagree on, as I think accuracy is only one aspect of utility, not the whole enchilada.
> The utility of a model is not arbitrary. Although it is not universal — it depends on the situtation — they are not *arbitrary*.
We agree. That is exactly the argument I made: "the situation (read: application, context, and all the other synonyms I used in my prior posts) defines the utility of a model". I'll go even further to say that exceeding the level of accuracy required for a specific need, **particularly at the cost of computational efficiency or other beneficial metrics**, is NOT useful, despite the gains in accuracy. Thus, model selection is not a function of accuracy alone, but of *accuracy, time limitations, acceptable tolerances, etc*.
> the utility of a model of reality is primarily *ALWAYS* connected to how well it approximates it. This is never an irrelevant aspect. To reject this is to reject science.
We agree here. I never claimed that the accuracy of the approximation was irrelevant. I said you can pick and choose which level of accuracy and computational efficiency you need when selecting a model depending on your application. That is not the same thing as saying "accuracy is bullshit and I reject science" or whatever you managed to interpret out of my prior comment. But note that you're now weaseling out of your previous claim by saying utility is "primarily always connected to" instead of "equivalent to" accuracy, as we can see in your first post in the thread. So overall you've agreed on all my points so far.
The one part where we may disagree is:
>It's obvious that the context in question is a scientific one, where one tries to approximate reality.
I agree science is ONE of many contexts in which statistical models may be used. However, alternatives include finance, controls, etc, where we may not need to model a set of aspects of reality, but just one, and maybe not even that accurately. In that case, even simple models can be highly effective and accurate for making decisions or other usecases (see: A random walk down Wallstreet, for example).
@johnabs In a scientific context, the concern is how well a model approximates reality. Period. This is not my “personal belief”, don't be ridiculous.
If two models have the same or comparable degrees of approximation, other aspects may come in consideration, but not otherwise.
This is the context not only of my post, but the one considered by Box.
I only made a further point that even in your extended case where models have non-scientific uses, even then you cannot decouple them from their approximative quality. Otherwise, the interpretation of Box's quote becomes “all models are wrong, but that is not important anyways”, which is definitely not what the quote was trying to convey.
The fact that this quote is so often misinterpreted is one reason why I dislike it. It's actually very poorly worded. It is *not* about abstract utility theory, as you have been interpreting it.
Can we please have a reasonable, polite discussion where we operate under the assumption that "the other person may be reasonable but acting under a different set of axioms, hence why they're drawing a different conclusion from my own"?
If you'd actually like a response to your post, I'm more than willing to continue the discussion, but if the comments about your views on my "deep confusion", "being ridiculous", and "projection" don't stop, I'm done. I have not acted in such a manner toward you, and I will no longer tolerate it.
Spoilers for if you decide to continue: this is not "abstract utility theory", this is about multi-criteria optimization, and the mathematical fact that vectors don't have an order unless someone imposes one, which is precisely what you are doing when making accuracy the primary/only metric by which you judge a model.
@johnabs No, this is your own projection. Read the link which states the context of the original quote.
It does not make sense to talk about “models being right or wrong” outside of a scientific context.
And this is definitely not what Box was referring to.