In my personal experience philosophy
of science rarely directly plays a role in how most scientists do research.
Here I’d like to explore ways in which your philosophy of science might
slightly shift how you weigh what you focus on when you do research. I’ll focus
on different philosophies of science (instrumentalism, constructive empiricism,
entity realism, and scientific realism), and explore how it might impact what
you see as the most valuable way to make progress in science, how much you
value theory-driven or data-driven research, and whether you believe your
results should be checked against reality.
We can broadly distinguish
philosophies of science (following Niiniluoto, 1999) in three main categories.
First, there is the view that there is no truth, known as anarchism. An example of this can be found in Paul Feyerabend’s
‘Against Method’ where he writes: Science is an essentially anarchic
enterprise: theoretical anarchism is more humanitarian and more likely to
encourage progress than its law-and-order alternatives” and “The only principle
that does not inhibit progress is: anything goes.” The second category contains
pragmatism, in which ‘truth’ is
replaced by some surrogate, such as social consensus. For example, Pierce (1878) writes: “The opinion
which is fated to be ultimately agreed to by all who investigate, is what we
mean by the truth, and the object represented in this opinion is the real. That
is the way I would explain reality.”. Rorty doubts such a final end-point of
consensus can ever be reached, and suggests giving up on the concept of truth
and talk about an indefinite adjustment of belief. The third category, which we
will focus on mostly below, consists of approaches that define truth as some
correspondence between language and reality, known as correspondence theories. In essence, these approaches adhere to a
dictionary definition of truth as ‘being in accord with fact or reality’. However,
these approaches differ in whether they believe scientific theories have a
truth value (i.e., whether theories can be true or false), and if theories have truth value, whether
this is relevant for scientific practice.
What do you think? Is anarchy the
best way to do science? Is there no truth, but at best an infinite updating of
belief with some hope of social consensus? Or is there some real truth that we
can get closer to over time?
Scientific
Progress and Goals of Science
It is possible to have different
philosophies of science because success in science is not measured by whether
we discover the truth. After all, how would we ever know for sure we discovered
the truth? Instead, a more tangible goal for science is to make scientific progress. This means we can
leave philosophical discussions about what truth is aside, but it means we will
have to define what progress in science looks like. And to have progress in
science, science needs to have a goal.
Kitcher (1993, chapter 4) writes: “One
theme recurs in the history of thinking about the goals of science: science ought to contribute to "the relief
of man's estate," it should enable us to control nature—or perhaps, where
we cannot control, to predict, and so adjust our behavior to an uncooperative
world—it should supply the means for improving the quality and duration of
human lives, and so forth.” Truth alone is not a sufficient aim for scientific
progress, as Popper (1934/1959) already noted, because then we would just limit
ourselves to positing trivial theories (e.g., for psychological science the
theory that ‘it depends’), or collect detailed but boring information (the
temperature in the room I am now in is 19.3 degrees). Kitcher highlights two
important types of progress: conceptual
progress and explanatory progress.
Conceptual progress comes from refining the concepts we talk about, such that
we can clearly specify these concepts, and preferably reach consensus about
them. Explanatory progress is improved by getting a better understanding of the
causal mechanisms underlying phenomena. Scientists will probably recognize the
need for both. We need to clearly define our concepts, and know how to measure
them, and we often want to know how things are related, or how to manipulate
things.
This distinction between conceptual
progress and explanatory progress aligns roughly with a distinction about
progress with respect to the entities
we study, and the theories we build
that explain how these entities are related. A scientific theory is defined as a
set of testable statements about the relation between observations. As noted
before, philosophies of science differ in whether they believe statements about
entities and theories are related to the truth, and if they are, whether this
matters for how we do science. Let’s discuss four flavors of philosophies of
science that differ in how much value they place in whether the way we talk
about theories and entities corresponds to an objective truth.
Instrumentalism
According to instrumentalism, theories
should be seen mainly as tools to solve practical problems, and not as truthful
descriptions of the world. Theories are instruments that generate predictions
about things we can observe. Theories often refer to unobservable entities, but
these entities do not have truth or falsity, and neither do the theories.
Scientific theories should not be evaluated based on whether they correspond to
the true state of the world, but based on how well they perform.
One important reason to suspend
judgment about whether theories are true or false is because of
underdetermination (for an explanation, see Ladyman, 2002). We often do not
have enough data to distinguish different possible theories. If it really isn’t
possible to distinguish different theories because we would not be able to
collect the required data, it is often difficult to say whether one theory is
closer to the truth than another theory.
From an instrumentalist view on
scientific progress, and assuming that all theories are underdetermined by
data, additional criteria to evaluate theories become important, such as simplicity. Researchers might use
approximations to make theories easier to implement, for example in
computational models, based on the convictions that simpler theories provide
more useful instruments, even if they are slightly less accurate about the true
state of the world.
Constructive
Empiricism
As opposed to instrumentalism,
constructive empiricism acknowledges that theories can be true or not. However,
it limits belief in theories only in
as far as they describe observable events. Van Fraassen, one of the main
proponents of constructive empiricism, suggests we can use a theory without
believing it is true when it is empirically
adequate. He says: “a theory is empirically adequate exactly if what it
says about the observable things and events in the world, is true”. Constructive
empiricists might decide to use a theory, but do not have to believe it is
true. Theories often make statements that go beyond what we can observe, but
constructive empiricists limit truth statements to observable entities. Because
no truth values are ascribed to unobservable entities that are assumed to exist
in the real world, this approach is grouped under ‘anti-realist’ philosophies
on science.
Entity
Realism
Entity
realists are willing to take one step beyond constructive empiricism and
acknowledge a belief in unobservable entities when a researcher can demonstrate
impressive causal knowledge of an unobservable) entity. When knowledge about an
unobservable entity can be used to manipulate its behavior, or if knowledge
about the entity can be used to manipulate other phenomena, one can believe
that it is real. However, researchers remain skeptical about scientific
theories.
Hacking
(1982) writes, in a very accessible article, how: “The vast majority of
experimental physicists are realists about entities without a commitment to
realism about theories. The experimenter is convinced of the existence of
plenty of "inferred" and "unobservable" entities. But no
one in the lab believes in the literal truth of present theories about those
entities. Although various properties are confidently ascribed to electrons,
most of these properties can be embedded in plenty of different inconsistent theories
about which the experimenter is agnostic.” Researchers can be realists about
entities, but anti-realists about models.
Scientific
Realism
We can compare the constructive
empiricist and entity realist views with scientific realism. For example,
Niiniluoto (1999) writes that in contrast to a constructive empiricist:
“a scientific realist sees theories
as attempts to reveal the true nature of reality even beyond the limits of
empirical observation. A theory should be cognitively successful in the sense
that the theoretical entities it postulates really exist and the lawlike descriptions
of these entities are true. Thus, the basic aim of science for a realist is
true information about reality. The realist of course appreciates empirical
success like the empiricist. But for the realist, the truth of a theory is a
precondition for the adequacy of scientific explanations.”
For
scientific realists, verisimilitude,
or ‘truthlikeness’ is treated as the basic epistemic utility of science. It is
based on the empirical success of theories. As De Groot (1969) writes: “The
criterion par excellence of true knowledge is to be found in the ability to
predict the results of a testing procedure. If one knows something to be true,
he is in a position to predict; where prediction is impossible, there is no
knowledge.” Failures to predict are thus very impactful for a scientific
realist.
Progress in
Science
There are more similarities than
differences between almost all philosophies of science. All approaches believe
a goal of science is progress. Anarchists refrain from specifying what progress
looks like. Feyerabend writes: “my thesis is that anarchism helps to achieve
progress in any one of the senses one care to choose” – but progress is still a
goal of science. For instrumentalists, the proof is in the pudding – theories
are good, as long they lead to empirical progress, regardless of whether these
theories are true. For a scientific realist, theories are better the closer the
more verisimilitude they have, or the closer the get to an unknown truth. For
all approaches (except perhaps anarchism) conceptual progress and explanatory
progress are valued.
Conceptual progress is measured by
increased accuracy in how a concept
is measured, and increased consensus
on what is measured. Progress concerning measurement accuracy is easily
demonstrated since it is mainly dependent on the amount of data that is
collected, and can be quantified by the standard error of the measurement. Consensus
is perhaps less easily demonstrated, but Meehl (2004) provides some
suggestions, such as a theory being generally talked about as a ‘fact’,
research and technological applications use the theory but there is no need to
study it directly anymore, and the only discussions of the theory at scientific
meetings are as in panels about history or celebrations of past successes. We
then wait for (and arguably arbitrary) 50 years to see if there is any change,
and if not, we consider the theory accepted by consensus. Although Meehl
acknowledged this is a somewhat brute-force approach to epistemology, he
believes philosophers of science should be less distracted by exceptions such
as Newtonian physics that was overthrown after 200 years, and acknowledge
something like his approach will probably work in practice most of the time.
Explanatory progress is mainly
measured by our ability to predict novel
facts. Whether prediction
(showing a theoretical prediction is supported by data) should be valued more
than accommodation (adjusting a
theory to accommodate unexpected observations) is a matter of
debate.
Some have argued that it doesn’t matter if a theory is stated before data is
observed or after data is observed. Keynes writes: “The peculiar virtue of
prediction or predesignation is altogether imaginary. The number of instances
examined and the analogy between them are the essential points, and the
question as to whether a particular hypothesis happens to be propounded before
or after their examination is quite irrelevant.” It seems as if Keynes
dismisses practices such as pre-registration, but his statement comes with a
strong caveat, namely that researchers are completely unbiased. He writes: “to
approach statistical evidence without preconceptions based on general grounds,
because the temptation to ‘cook’ the evidence will prove otherwise to be
irresistible, has no logical basis and need only be considered when the
impartiality of an investigator is in doubt.”
Keynes’ analysis of prediction
versus accommodation is limited to the evidence in the data. However, Mayo (2018)
convincingly argues we put more faith in predicted findings than accommodated
findings because the former have passed a
severe test. If data is used when generating a hypothesis (i.e., the
hypothesis has no use-novelty) the
hypothesis will fit the data, no matter whether the theory is true or false. It
is guaranteed to match the data, because the theory was constructed with this
aim. A theory that is constructed based on the data has not passed a severe
test. When novel data is collected in a well-constructed experiment, a
hypothesis is unlikely to pass a test (e.g., yield a significant result) if the
hypothesis is false. The strength from not using the data when constructing a
hypothesis comes from the fact that is has passed a more severe test, and had a
higher probability to be proven wrong (but wasn’t).
Does Your
Philosophy of Science Matter?
Even if scientists generally agree
that conceptual progress and explanatory progress are valuable, and that
explanatory progress can be demonstrated by testing theoretical predictions,
your philosophy of science likely influences how much you weigh the different
questions researchers ask when they do scientific research. Research can be
more theory driven, or more exploratory, and it seems plausible your views on
which you value more is in part determined by your philosophy of science.
For example, do you perform research
by formalizing strict theoretical predictions, and collect data to corroborate
or falsify these predictions to increase the verisimilitude of the theory? Or
do you largely ignore theories in your field, and aim to accurately measure
relationships between variables? Developing strong theories can be useful for a
scientific field, because they facilitate the organization of known phenomena, help
to predict what will happen in new situations, and guide new research.
Collecting reliable information about phenomena can provide the information
needed to make decisions, and provides important empirical information that can
be used to develop theories.
For a scientific realist a main aim
is to test whether theories reflect reality. Scientific research starts with
specifying a falsifiable theory. The goal of an experiment is to test the
theory. If the theory passes the test, the theory gains verisimilitude, if it
fails a test, it loses verisimilitude, and needs to be adjusted. If a theory
repeatedly fails to make predictions (what Lakatos calls a degenerative research line) it is eventually abandoned. If the
theory proves successful in making predictions, it becomes established
knowledge.
For an entity realist like Hacking
(1982), experiments provide knowledge about entities, and therefore experiments
determine what we believe, not theories. He writes: “Hence, engineering, not
theorizing, is the proof of scientific realism about entities.” Van Fraassen similarly
stresses the importance of experiments, which are crucial in establishing facts
about observable phenomena. He sees a role for theory, but it is quite
different of the role it plays in scientific realism. Van Fraassen writes: “Scientists
aim to discover facts about the world—about the regularities in the observable
part of the world. To discover these, one needs experimentation as opposed to
reason and reflection. But those regularities are exceedingly subtle and
complex, so experimental design is exceedingly difficult. Hence the need for
the construction of theories, and for appeal to previously constructed theories
to guide the experimental inquiry.”
Theory-driven
versus data-driven
One might be tempted to align
philosophies of science along a continuum of how strongly theory driven they
are (or confirmatory), and how
strongly data-driven they are (or exploratory).
Indeed, Van Fraassen writes: “The phenomenology of scientific theoretical
advance may indeed be exactly like the phenomenology of exploration and discovery
on the Dark Continent or in the South Seas, in certain respects.” Note that
exploratory data-driven research is not void of theory – but the role theories
play has changed. There are two roles, according to Fraassen. First, the outcome
of an experiment is ‘filling in the
blanks in a developing theory’. The second role theories play is in that,
as the regularities we aim to uncover become more complex, we need theory to
guide experimental design. Often a theory states there must be something, but it is very unclear what
this something actually is.
For example, a theory might predict
there are individual differences, or contextual moderators, but the scientist
needs to discover which individual differences, or what contextual moderators.
In this instance, the theory has many holes in it that need to be filled. As
scientists fill in the blanks, there are typically new consequences that can be
tested. As Fraassen writes: “This is how experimentation guides the process of
theory construction, while at the same time the part of the theory that has already
been constructed guides the design of the experiments that will guide the
continuation”. For example, if we learn that as expected individual differences
moderate an effect, and the effect is more pronounced for older compared to
younger individuals, these experimental results guide theory construction. Fraassen
goes as far as to say that “experimentation
is the continuation of theory construction by other means.”
For a scientific realist experimentation
has the main goal to test theories, not to construct them. Exploration is still
valuable but is less prominent in scientific realism. If a theory is at the
stage where it predicts something will happen, but is not specific about what
this something is, it is difficult to come up with a result that would falsify
that prediction (except for cases where it is plausible that nothing would
happen, which might be limited to highly controlled randomized experiments).
Scientific realism requires well-specified theories. When data do not support
theoretical predictions, this should be consequential. It means a theory is
less ‘truth-like’ than we thought before.
Subjective
or Objective Inferences?
Subjective
beliefs in a theory or hypothesis play an important role in science. Beliefs
are likely to have strong motivational power, leading scientists to invest time
and effort in examining the things they examine. It has been a matter of debate
whether subjective beliefs should play a role in the evaluation of scientific
facts.
Both
Fisher (1935) as Popper (1934/1959) disapproved of introducing subjective
probabilities into statistical inferences. Fisher writes: “advocates of inverse
probability seem forced to regard mathematical probability, not as an objective
quantity measured by observed frequencies, but as measuring merely
psychological tendencies, theorems respecting which are useless for scientific
purposes.” Popper writes: “We must distinguish between, on the one hand, our subjective experiences or our feelings
of conviction, which can never justify any statement (though they can be made
the subject of psychological investigation) and, on the other hand, the objective logical relations subsisting
among the various systems of scientific statements, and within each of them.”
For Popper, objectivity does not reside in theories, which he believes are
never verifiable, but in tests of
theories: “the objectivity of scientific statements lies in the fact that
they can be inter-subjectively tested.”
This
concern cuts across statistical approaches. Taper and Lele (2011) who are
likelihoodists write: We dismiss Bayesianism for its use of subjective priors
and a probability concept that conceives of probability as a measure of
personal belief.” They continue: “It is not that we believe that Bayes' rule or
Bayesian mathematics is flawed, but that from the axiomatic foundational
definition of probability Bayesianism is doomed to answer questions irrelevant
to science. We do not care what you believe, we barely care what we believe,
what we are interested in is what you can show.” Their approach seems closest
to a scientific realism perspective. Although they acknowledge all models are
false, they also write: “some models are better approximations of reality than
other models” and “we believe that growth in scientific knowledge can be seen
as the continual replacement of current models with models that approximate
reality more closely.”
Gelman
and Shalizi, who use Bayesian statistics but dislike subjective Bayes, write: “To
reiterate, it is hard to claim that the prior distributions used in applied
work represent statisticians' states of knowledge and belief before examining
their data, if only because most statisticians do not believe their models are
true, so their prior degree of belief in all of Θ [the parameter space used to
generate a model] is not 1 but 0”. Although subjective Bayesians would argue
that having no belief that your models are true is too dogmatic (it means you
would never be convinced otherwise, regardless of how much data is collected)
it is not unheard of in practice. Physicists know the Standard Model is wrong,
but it works. This means they assign a probability of 0 to the standard model
being true – which violates one of the core assumptions of Bayesian inference
(which is that the plausibility assigned to a hypothesis can be represented as
a number between 0 and 1). Gelman and Shalizi approach statistical inferences
from a philosophy of science perhaps closest to constructive empiricism when
they write: “Either way, we are using deductive reasoning as a tool to get the most
out of a model, and we test the model - it is falsifiable, and when it is
consequentially falsified, we alter or abandon it.”
Both
Taper and Lele (2011) as Gelman and Shalizi (2013) stress that models should be
tested against reality. Taper and Lele want procedures that are reliable (i.e.,
are unlikely to yield incorrect conclusions in the long run), and that provide
good evidence (i.e., when the data is in, it should provide strong relative
support for one model over another). They write: “We strongly believe that one
of the foundations of effective epistemology is some form of reliabilism. Under
reliabilism, a belief (or inference) is justified if it is formed from a
reliable process.” Similarly, Gelman and Shalizi write: “the hypothesis linking
mathematical models to empirical data is not that the data-generating process
is exactly isomorphic to the model, but that the data source resembles the
model closely enough, in the respects which matter to us, that reasoning based
on the model will be reliable.”
As
an example of an alternative viewpoint, we can consider a discussion about
whether optional stopping (repeatedly analyzing data and stopping the data
analysis whenever the data supports predictions) is problematic or not. In
Frequentist statistics the practice of optional stopping inflates the error
rate (and thus, to control the error rate, the alpha level needs to be adjusted
when sequential analyses are performed). Rouder (2014) believes optional
stopping is no problem for Bayesians. He writes: “In my opinion, the key to understanding
Bayesian analysis is to focus on the degree of belief for considered models,
which need not and should not be calibrated relative to some hypothetical truth.”
It is the responsibility of researchers to choose models they are interested in
(although how this should be done is still a matter of debate). Bayesian
statistics allows researchers to update their belief concerning these models
based on the data – irrespective of whether these models have a relation with
reality. Although optional stopping increases error rates (for an excellent
discussion, see Mayo, 2018), and this reduces the severity of the tests the
hypotheses pass (which is why researchers worry about such practices
undermining reproducibility) such concerns are not central to a subjective
Bayesian approach to statistical inferences.
Can You
Pick Only One Philosophy of Science?
Ice-cream
stores fare well selling cones with more than one scoop. Sure, it can get a bit
messy, but sometimes choosing is just too difficult. When it comes to
philosophy of science, do you need to pick one approach and stick to it for all
problems? I am not sure. Philosophers seem to implicitly suggest this (or at
least they don’t typically discuss the pre-conditions to adopt their proposed
philosophy of science, and seem to imply their proposal generalizes across
fields and problems within fields).
Some
viewpoints (such as whether there is a truth or not, and if theories have some
relation to truth) seem rather independent of the research context. It is still
fine to change your view over time (philosophers of science themselves change
their opinion over time!) but they are probably somewhat stable.
Other
viewpoints seem to leave more room for flexibility depending on the research
you are doing. You might not believe theories in a specific field are good
enough to be used as anything but crude verbal descriptions of phenomena. I
teach an introduction to psychology course to students at the Eindhoven
Technical University, and near the end of one term a physics student approached
me after class and said: “You very often use the word ‘theory’, but many of
these ‘theories’ don’t really sound like theories”. If you have ever tried to
create computational models of psychological theories, you will have
experienced it typically cannot be done: Theories lack sufficient detail.
Furthermore, you might feel the concepts used in your research area are not
specified enough to really know what we are talking about. If this is the case,
you might not have the goal to test theories (or try to explain phenomena) but mainly want to focus on conceptual progress
by improving measurement techniques or accurately estimate their effect sizes.
Or you might work on more applied problems and believe that a specific theory
is just a useful instrument that guides you towards possibly interesting
questions, but is not in itself something that can be tested, or that
accurately describes reality.
Conclusion
Researchers often cite and use
theories in their research, but they are rarely explicit about what these
theories mean to them. Do you believe a theory reflects some truth in the world,
or are they just useful instruments to guide research that should not be
believed to be true? Is the goal of your research to test theories, or to
construct theories? Do you have a strong belief that the unobservable entities
you are studying are real, or do you prefer to limit your belief to statements
about things you can directly observe? Being clear about where you stand with
respect to these questions might make it clear what different scientists expect
scientific progress should look like and clarify what their goals are when they
collect data. It might explain differences in how people respond when a
theoretical prediction is not confirmed, or why some researchers prefer to
accurately measure the entities they study, while others prefer to test
theoretical predictions.
References
Feyerabend, P. (1993). Against
method. London: Verso.
Fisher, R. A. (1935). The design of
experiments. Oliver And Boyd; London.
Gelman, A., & Shalizi, C. R.
(2013). Philosophy and the practice of Bayesian statistics. British Journal of
Mathematical and Statistical Psychology, 66(1), 8–38. https://doi.org/10/f4k2h4
Keynes, J. M. (1921). A Treatise on
Probability. Cambridge University Press.
Kitcher, P. (1993). The advancement
of science: science without legend, objectivity without illusions. New York:
Oxford University Press.
Ladyman, J. (2002). Understanding
philosophy of science. London ; New York: Routledge.
Mayo, D. G. (2018). Statistical
inference as severe testing: how to get beyond the statistics wars. Cambridge:
Cambridge University Press.
Meehl, P. E. (2004). Cliometric
metatheory III: Peircean consensus, verisimilitude and asymptotic method. The British Journal for the Philosophy of
Science, 55(4), 615–643.
Niiniluoto, I. (1999). Critical
Scientific Realism. Oxford University Press.
Popper, K. R. (1959). The logic of
scientific discovery. London; New York: Routledge.
Rouder, J. N. (2014). Optional
stopping: No problem for Bayesians. Psychonomic Bulletin & Review, 21(2),
301–308.
Taper, M. L., & Lele, S. R.
(2011). Evidence, Evidence Functions, and Error Probabilities. In P. S.
Bandyopadhyay & M. R. Forster (Eds.), Philosophy of Statistics (Vol. 7, pp.
513–532). Amsterdam: North-Holland. https://doi.org/10.1016/B978-0-444-51862-0.50015-0
