Is Science Trustworthy? Part 1 - The Scientific Method

This is the first in a series of essays on science and what makes it variably reliable. Future essays will cover measurement, experiment, theory, scientists, social structures, and institutions.

A Philosopher Lecturing on the Orrery by Joseph Wright of Derby, 1766

Introduction

Science has, throughout history, proven to be the most powerful method of inquiry into the natural world ever devised. Its success has enabled it to outcompete other methods of inquiry like mysticism, free association, revelation, and rationalism. Science proceeds through application of the scientific method, using experimentation to collect data, which is then understood through the lens of scientific theories. This can be thought of as the fundamental framework of science. These theories and data are produced by individual scientists cooperating through a sociological incentive structure, often funded and directed by scientific and research institutions. This can be understood as the human factor in science – how science actually proceeds in the real world. While the fundamental framework of science can enable the creation of the most reliable possible knowledge of the world, the influence of the human factor causes it to often fall short of this lofty goal.

Science is, however, rarely presented in these terms. The accounts of science that are found in popular reporting or in textbooks often paint an idealized picture of science that little resembles how it develops in practice. In fact, some scientists have argued that this should be the case. In Should History of Science be Rated X? historian of science Stephen G. Brush writes:

“I will examine arguments that young and impressionable students at the start of a scientific career should be shielded from the writings of contemporary science historians for reasons … [such as], that these writings do violence to the professional ideal and public image of scientists as rational, open-minded investigators, proceeding methodically, grounded incontrovertibly in the outcome of controlled experiments, and seeking objectively for the truth, let the chips fall where they may.“^[1]

The public image of scientists so described stems most saliently from hagiographic portrayals of historical figures like Galileo, Newton, and Mendel - sanitized portrayals that often omit the propaganda^[2], equation fudging^[3], and data falsification^[4] they employed to sell their ideas. Consequently, the majority of people – and even scientists themselves – hold misguided views about how science really works.

This misguided view is reflected in some of our conceptions of scientific literacy, which is often treated as if it were simply the knowledge of a particular assemblage of facts¹. For instance, a typical question on the NSF test of scientific literacy is the true or false question "Antibiotics kill viruses as well as bacteria." But simply knowing the commonly accepted facts does not imply understanding. It does not reflect any knowledge of how those facts were discovered, the context in which the discovery took place, or why these facts should be considered more reliable than the supposed facts they superseded. Indeed, scientific literacy as assessed in this way barely even correlates with disbelief in pseudoscientific concepts like lucky numbers^[5]. Regardless of how well such questions measure scientific literacy, it remains that only 50% of Americans knew that antibiotics don't work on viruses - and that out of 11 countries tested this result was among the best.^[6] Given this state of affairs, many officials and pundits have claimed that it is dangerous for the public to try to think for themselves about scientific matters^[7] – this is reflected in the slogans demanding we “trust the science” or “listen to the experts” that are often bandied about in political discourse today.

Many people, of course, do not trust science - and they have reason not to. Scientists are just as likely to be beholden to financial interests as anyone else^[8], and can bring their personal biases into their research^[9]. Across cultures and throughout history, powerful institutions have sought to control and manipulate science. Disagreements abound between practicing scientists themselves, with disputes as fundamental as whether another researcher’s field is even real science. Among the American public trust in science is more polarized than ever^[10], with a 30-point spread between Democrats and Republicans^[11]. Likely this is due to the academy being mostly left-wing^[12] - if the academy were instead right-leaning then liberals might find themselves on the less-trusting side of the split. All of these factors - conflicts of interest, institutional backing, research methods, and political bias - can affect how trustworthy science is, or if it can even be trusted at all.

This is crucial because trust is actually a necessary ingredient in science, and in fact in any advanced knowledge production^[13]. No scientist can independently verify all of the past results on which their work is based. No scientist has the time to carefully read through all the papers published in their field, let alone verify the citations are correct, let alone replicate the data analysis for themselves. After a certain point one must rely on well-calibrated trust (hopefully not blind faith) in order to make further progress. For a scientist, miscalibrated trust can lead to basing work on fraudulent results, or worse, performing their life's work under the auspices of a research tradition that is later discredited. For the public, miscalibrated trust can lead to choosing medical treatments that make them sicker rather than better, or supporting political initiatives that do more harm than good. Given the central importance of trust, this series of essays examines whether, and to what extent, science is trustworthy, by closely examining both the fundamental framework of science and the human factors in science.

History of the Scientific Method

Science is a method of inquiry into the natural world, and the scientific method describes the steps by which should be performed. Apart from procedural guidelines, the scientific method also assumes certain implicit metaphysical principles, which must be understood. Psychologist and historian Christopher Green writes “all research methods require philosophical justification at some level … If they are not made explicit … such positions will be held uncritically and be propagated to others by insinuation rather than direct argument.”^[14] The processes and presuppositions can be illustrated by examining the evolution of science. The objective is not to provide a full account of the history of the scientific method or to detail the opinions of various thinkers, but to situate the present understanding of science within a historical progression.

Although protoscientific inquiry was conducted by the Egyptians and Babylonians in medicine and astronomy, it was the Greeks, and especially Aristotle, who developed the scientific method into a form that would be recognizable today. The pre-Socratic philosopher Thales understood phenomena as events with causes explicable by nature, and was consequently the first person to posit non-supernatural explanations for things like lightning, earthquakes, and eclipses. The position that phenomena have natural causes is naturalism, one of the metaphysical foundations of science. The requirement for naturalism in science means that supernatural explanations are inherently unscientific and outside the purview of scientific inquiry. That is not to say that things previously thought to be supernatural cannot be later be explained scientifically, but these explanations must be based on nature and nature’s laws.

Aristotle offered the first formal description of scientific knowledge in his Posterior Analytics, part of a larger collection of works on logical analysis called the Organon. He used particulars (observations) to derive knowledge of universals (universal truths) through syllogistic reasoning. The thus-derived universals could then be checked against particulars in repeating cycles to advance scientific knowledge. Already in Aristotle’s work several key facets of the scientific method are demonstrated: logic, empiricism, deduction, and induction. Logic, or internal logical consistency, is the stipulation that all claims should be consistent with each other, without contradiction. If one scientific claim contradicts another, one or both of them must necessarily be wrong, and hence not be an example of scientific knowledge. Internal logical consistency is a requirement for any kind of knowledge, not merely the scientific type. Empiricism means that claims of scientific knowledge should be externally correspondent, that is they should be supported by evidence observable in the real world. Empiricism is required for scientific knowledge, however it is not required for some other types of knowledge like mathematical knowledge.

The other two facets are induction and deduction, which are opposites in terms of the direction of reasoning. Induction is the derivation of general principles from individual observations. Whether this is warranted or indeed possible has been a topic of discussion among philosophers for centuries, but it is generally regarded as integral to science. Indeed, rejecting induction undermines not just science but also common sense knowledge, and so can be considered a form of radical skepticism that is inherently incompatible with science.^[15] Deduction is the elaboration of that which is necessarily entailed by a set of axioms or presuppositions. If a hypothesis is true, what must that also imply? For instance, if a theory states that all substances are composed of atoms, then atoms should be found when studying a novel substance.

For Aristotle, scientific knowledge required not just the demonstration of universal truths, but knowledge of their causes as well.² Therefore while induction was sufficient to discover universals through a process of abstraction, deduction from existing truths was necessary to establish causes. For this reason, Aristotle described intuition as the originative source of scientific knowledge, but not itself a form of scientific knowledge. As we will see, this distinction between the process through which scientific knowledge originates, and scientific knowledge in its mature form, is an important distinction in the philosophy of science. Aristotle’s focus on understanding causes remains important today, although explanation through syllogism is no longer deemed sufficient.

Aristotle’s other critical contribution to the philosophy of science lay in his commitment to realism, in contrast with the idealism of his teacher Plato. The difference between the two metaphysics is what they consider reality: in idealism, reality exists in the mind as a mental construct related to ideas, while realism contends that there is an objective reality outside of the observer. This distinction can be seen in their differing approaches to Plato’s theory of Forms. For Plato, the Forms were abstract objects grasped by pure reason, and constituted the “real” world, while the material world was relegated to being a imperfect reflection of the world of the Forms. Aristotle rejected the disjunction between the Forms and the material world, considering that every Form was the form of a thing which could be investigated by studying the natural world. As science is a means of inquiry into the natural world, it must be based on realism, since scientific methods cannot be used to interrogate abstract objects outside of nature (the so-called formal sciences like logic, mathematics, and theoretical linguistics have no such limitation, and are not considered sciences for the purposes of this essay).

After the classical period the scientific method continued to be refined during the Islamic golden age, with a greater emphasis on using experimentation to come to conclusions about the world. The Islamic scholar Ibn al-Haytham used extensive experimentation in order to develop a modern theory of vision, demonstrating that the ancient emission theory of vision (wherein the eyes emit rays of light for seeing) was incorrect. Al-Haytham also advanced an early form of scientific parsimony, saying that “the extramission of [visual] rays is superfluous and useless.”^[16] Scientific parsimony, the idea that the simplest explanation that can explain all the facts should be preferred, is famously exemplified by Occam’s razor, “Entities must not be multiplied beyond necessity”.

Scientific thinking began to reemerge in Europe as the continent moved out of the medieval period and Greek and Arabic texts were translated into Latin. In the 13th century, Roger Bacon described the scientific method as a repeating cycle of observation, hypothesis, and experimentation requiring independent replication. The translation and discussion of classical texts continued in Europe throughout the Renaissance, and served as the intellectual jumping-off point for the scientific revolution, which began in 1543 when Copernicus published his heliocentric model of the solar system. Scientists in this period began to go beyond the classical philosophers in a significant way, and their ideas were often formed in reaction to the Aristotelian framework. Many of the norms and standards of modern science were developed during this time.

One of the most important developments in the philosophy of science during this period was Francis Bacon's 1620 publication of his Novum Organum, which, as the name suggests, was a direct response to Aristotle. In it, Bacon proposes an early modern reformulation of the scientific method, based on experimentation and induction. Bacon also vocally eschewed teleology in science:

"[H]uman understanding, incapable of resting, seeks for something more intelligible. Thus, however, while aiming at further progress, it falls back to what is actually less advanced, namely, final causes; for they are clearly more allied to man’s own nature, than the system of the universe, and from this source they have wonderfully corrupted philosophy."^[17]

Unlike theology and some branches of philosophy, science is not teleological. That is, science does not assume or require that natural phenomena have any inherent purpose or goal (final cause). For instance, an explanation of human origins from the perspective of Christian theology may begin with the teleological statement that humans were created to love and serve God. A scientific account will instead describe the natural processes that shaped humans into their modern form, without ascribing any intent or purpose to these processes.

The emphasis on quantitative reasoning in science also solidified in the 17th century. In 1623 Galileo published Il Saggiatore (The Assayer), wherein he states that:

“[The universe] cannot be understood unless one first learns to comprehend the language and interpret the characters in which it is written. It is written in the language of mathematics, and its characters are triangles, circles, and other geometrical figures, without which it is humanly impossible to understand a single word of it; without these, one is wandering around in a dark labyrinth.”^[18]

With this work Galileo is famous for declaring mathematics (geometry, specifically) as the language of science. Galileo was motivated by a mathematically idealist intellectual tradition inherited from the Greeks that saw the universe as constructed according to geometric principles, and which was satisfied with discovering the exact mathematical relations that ordered it. This represents a notable departure from the Aristotelian focus on causes, as while mathematics can be used to describe phenomena it cannot be used to explain them. At the same time, mechanical philosophy was attempting to purge occultism from science by explaining phenomena mechanistically in terms of matter and motion; Cartesian mind-body dualism served to excise all non-material influence from natural phenomena so that they could be studied scientifically.^[19] Unfortunately, neither Galileo nor Kepler were able to explain the causes behind the mathematical relations they discovered, and the mechanistic theories proposed by Descartes and others were not able to produce the correct mathematical relations. The synthesis came from Newton, who proposed the law of universal gravitation - an attractive force between all matter - as the fundamental principle behind both Galileo's kinematics and Kepler's laws of planetary motion. Newton's synthesis was resisted by other prominent scientists in the mechanistic tradition, who saw forces acting at a distance as a reintroduction of the occult into science. Newton replied:

"These Principles I consider, not as occult Qualities, supposed to result from the specifick Forms of Things, but as general Laws of Nature, by which the Things themselves are formed; their Truth appearing to us by Phaenomena, though their Causes be not yet discovered. For these are manifest Qualities, and their Causes only are occult."^[20]

Thus Newton posited gravity as the cause of terrestrial and celestial phenomena, but left the mechanistic cause of gravity itself to be discovered. While this justification was not considered acceptable to his critics, it was ultimately the ostensible predictive accuracy of Newton's physics that lead to its success.^[3]

Newton's work also served to provide a strong argument for the epistemological primacy of empiricism over rationalism, the debate over which was a central philosophical question of the time. Rationalism, most famously championed by Descartes, held that reason is the primary source of knowledge, and that certain truths could be known a priori, independent of experience. Empiricism, as argued by Locke and Hume, instead held that sensory experience was the primary source of knowledge, and that we can only know what we observe. This debate in some ways recapitulates the debate between Plato and Aristotle, with the world-independent a priori truths of rationalism recalling Plato’s idealism, and the observationally-derived truths of empiricism recalling Aristotle’s realism. A synthesis of these two positions was provided by Immanuel Kant in his Critique of Pure Reason, which can be summed up in the quote “thoughts without content are empty, [sensory] intuitions without concepts are blind”.^[21] In the context of this essay, scientific knowledge in the form of theories relies on both reason to ensure logical non-contradiction, and empiricism to demonstrate correspondence with the external world.

In the 19th century the field of scientific statistics developed out of the study of measurement errors. In the latter half of the century the advent of new statistical techniques opened up new vistas of scientific discovery in psychology and social science. These fields involve complex systems with high causal density (i.e. the number of factors that can cause variation in outcomes is very large and may not be exhaustively enumerable) and are not amenable to description by statements of universal governing laws. Instead, statistics enables the scientific discovery of probabilistic and correlative statements that maintain predictive power despite lacking the fidelity of first-principle laws. Statistics has since become an essential tool in all fields of science, and will be discussed in more depth in the essay on empirical evidence.

By the time of the 20th century the veritable explosion of scientific inquiry had led to the question of demarcation: how could science be distinguished from non-science? The logical positivist school held that the only meaningful scientific statements were those that could be verified logically or empirically. Karl Popper opposed verificationism and instead held that scientific theories could never be verified, only falsified, and that therefore falsifiability was the true criterion of demarcation. Popper pointed out that no amount of evidence can ever definitively prove a theory, but a single observation could potentially falsify it. Since we do not have access to all possible data, it is impossible to determine if at some future point new evidence might arise that would contradict a prevailing theory. Therefore, unlike a mathematical proof which may be deduced purely from stated axioms, a scientific theory can never be considered verified or proven. Instead, scientific theories must be formulated so that they are falsifiable, and subjected to empirical tests that attempt to falsify them. Any theories that survive falsification can be considered potentially true, since they have yet to be proven false. As it would be impossible to genuinely falsify a true statement through empirical observation, this method would falsify all theories that did not fulfill the requirement for external correspondence, leaving only those theories that did in fact map onto reality.

The most recent addition to the requirements of science is operationalism, which was formulated most influentially by physicist Percy Bridgman in his 1927 book The Logic of Modern Physics. Bridgman’s central insight, inspired by Einstein’s relativity, is that concepts are scientifically meaningless unless a series of operations can be performed to measure them.³ One of the successes of Einsteinian physics was providing an operational definition of simultaneity: if two photons emitted from two locations arrive at the midpoint of those locations at the same time, then the emission of those photons (tied to whatever event) can be said to be simultaneous. By proposing an operational definition, the concept of simultaneity was stripped of any ambiguity or intuitive loading, and became testable and repeatable. This represented an important shift away from Newton, who said “I do not define Time, Space, Place or Motion, as being well known to all.”^[22] The test of operational construction is important to ensure that a concept is well defined and precise, removing the ambiguity of merely verbal constructions. Proper operationalization should eliminate discretion – that is that the operation can be performed algorithmically with no personal judgment required, in much the same way as measuring a piece of paper with a ruler requires no subjective judgment to obtain a result.

Prediction and Explanation

It should be clear from the historical analysis that science has two roles: to predict, and to explain. At a minimum science should make accurate predictions. Predictions may be mathematical, such as that the pressure and volume of a gas obeys a particular equation, or existential, such as that when an aqueous solution of copper sulfate is mixed with zinc it will react and produce a copper precipitate. A mature science should also be able to explain the predictions by way of causal theories. To understand the difference between prediction and explanation, consider the Hazen-Williams equation, which relates the pressure drop along a pipe to the physical characteristics of the pipe:

The equation has excellent predictive power and is widely used in engineering, however it offers no explanation for why the variables should be arranged as they are. It is a purely heuristic (though accurate) approximation that cannot be derived from theory or first principles. Compare that to the inverse-square law which describes how radiation from a point source is inversely proportional to the square of the distance to the source:

\( I = \frac{P}{A} = \frac{P}{4\pi r^2}\)

It is both predictive and derivable from first principles in spatial mechanics, meaning it can offer a real explanation for why the result is meaningful. As with Kepler’s laws, it is common for scientific facts to begin as empirical relationships, only later being explained by theory - the alternative being that a theory predicts an phenomenon which is then observed. It is important to note that a theory purporting to explain an empirical relationship may turn out to be incorrect or incomplete – it will then be superseded by a superior theory. This means that the explanation was illusory, however it does not change the validity of the observed relationship.

In order for a theory to explain an observed empirical relationship, it must offer a mechanism by which the relationship arises. When speaking of quantitative predictions, a mathematical formula is either a direct expression of a mechanistic theory, or it is a heuristic. This would imply that Newtonian gravity, lacking a known causal mechanism, is a heuristic. This is indeed true, and in fact to this day no mechanistic theory of gravity⁴ has been proposed which has attracted significant research interest. One problem with non-mechanistic theories is that they cannot be known to have the correct number of degrees of freedom. This is aptly demonstrated by a quote from a meeting between physicists Freeman Dyson and Enrico Fermi:

In desperation I asked Fermi whether he was not impressed by the agreement between our calculated numbers and his measured numbers. He replied, “How many arbitrary parameters did you use for your calculations?” I thought for a moment about our cut-off procedures and said, “Four.” He said, “I remember my friend Johnny von Neumann used to say, with four parameters I can fit an elephant, and with five I can make him wiggle his trunk.”^[23]

Without a clear picture of the mechanistic processes underlying a predictive equation, it is not possible to know if the equation has been overfit to observed data (or whether the parameters need to be decomposed to expose more fundamental mechanics). It is also very difficult to unify heuristics when there is no underlying mechanistic system to guide the synthesis - one recalls that there is no accepted unification of gravity and electromagnetism.

The types of mechanisms that are acceptable will vary by field. In general, as one ascends the hierarchy of complexity from the physical to social sciences (from physics, to chemistry, to biology, to psychology, and so on) the mechanisms used in a higher-order science should be reducible to mechanisms at a more fundamental level, although a higher-order science does not warrant that the mechanistic explanations in the levels below it are complete or correct. In physics, the accepted mechanisms going back to the 17th century are matter, motion, and forces transferred by physical contact. The 19th century saw the emergence of field theory as a descriptive mathematics of electromagnetism, but without an underlying mechanistic explanation of the fields, they are not themselves mechanical. In chemistry, mechanistic explanations employ atomic theory and chemical bonding. These theories evolved in the 19th and early 20th centuries out of the non-mechanistic theory of chemical affinities, described by Newton as a "certain secret principle in nature by which liquors are sociable to some things and unsociable to others."^[24] In biology, the characteristics of biological materials and biological processes are used. An example of the former is the conduction of electrical signals by nerves, which replaced Galvani's non-mechanistic theory that living tissues contained a vital force that could respond to electricity. An example of the latter is the theory that maggots arise in rotten meat because flies lay eggs in it. This explanation superseded the non-mechanistic theory of spontaneous generation of life from non-living material.⁵ In psychology mechanistic explanations invoke the behaviour of neural circuits and brain regions. For any social science operating on a level at which the mind is taken for granted, there is so much causal density that the difficulty is not so much in coming up with a mechanism that might produce the observed phenomena as it is in determining which proposed mechanisms are actually causally related to the phenomena at hand.

While mechanisms are required to explain phenomena, it is not guaranteed that having a full mechanistic understanding of a system will enable straightforward predictions of the behaviour of the system. For instance, fluid dynamics is governed by mechanically-based conservation laws (conservation of momentum, of mass, etc.) that are instantiated mathematically in the Navier-Stokes equations. However, to predict how a fluid system with turbulent flow⁶ will evolve over time it is necessary to implement the Navier-Stokes equations in a computational model and conduct detailed simulations. This is due to the problem of computational irreducibility. A system is computationally irreducible if it is not possible to calculate an arbitrary future state from the current state without calculating all states in between^[25]. That is, it is not possible to skip ahead in computation: all intervening steps must be computed in order to determine the future state. We can contrast this with a computationally reducible system, like that of the positions of the planets. We can know where Jupiter will be in a year's time by solving the relevant equations. It was precisely the computational reducibility of planetary positions that enabled the prediction and discovery of Neptune in 1846. However, the other side of the reducibility coin means that many empirical relationships like the Hazen-Williams equation cannot be explained by a first principles theory, as they are coarse approximations of the underlying complexity. This is even more true of predictions arising from data-driven models like neural networks, which may be very accurate, but offer little in the way of explanatory power that can be used to scaffold a mechanistic theory.⁷

Pseudoscience

The history of science demonstrates its foundational reliance on metaphysical realism and naturalism. Its methodology involves a systematic approach that employs observation, hypothesis formulation, experimentation, and analysis that must be independently replicable. Within its framework, science engages both inductive reasoning, moving from observations to theories, and deductive reasoning, predicting observations from existing theories. It also adheres to the principle of parsimony, favoring the simplest explanations to prevent overfitting to data. Scientific knowledge must be internally logically consistent and externally correspondent. Observations in science must be operationalized, while theories should be both mechanistic and causal in nature, to ensure the existential possibility of scientific propositions. Rather than being unequivocally proven or verified by experimentation, candidates for scientific knowledge are subject to falsification via an adversarial process that prunes false conclusions. When this process is followed correctly, the result is reliable knowledge that is true regardless of one's perspective or opinions.^[26]

However, not every field that has been given the label of science historically or in the modern day meets these strict standards. Psychoanalysis, the school of psychology invented by Freud, was famously targeted by Popper as being unfalsifiable. Any behaviour could be explained through psychoanalysis, and so it made no claims that could potentially be used to falsify it. Accordingly, Popper considered it a pseudoscience. Speaking of Freudian and Adlerian psychology, Popper said “It was precisely this fact – that they always fitted, that they were always confirmed – which in the eyes of their admirers constituted the strongest argument in favour of these theories. It began to dawn on me that this apparent strength was in fact their weakness.”^[27]

Early psychology was generally pseudoscientific due to its inability to offer operational definitions for the constructs it was purporting to measure. For instance, William James was a founding figure in American psychology, and in 1890 he published The Principles of Psychology. In the book he describes the “tip of the tongue” phenomenon, however he describes it based on introspective accounts from himself and others. Psychology first differentiated itself from philosophy as an experimental discipline under one of James’ contemporaries, Wilhelm Wundt. Wundt, considered the father of experimental psychology, opened the first psychology lab in 1879. Despite his pioneering efforts in introducing experimentation into psychology, his work still fell short of proper operational standards. For instance, one of Wundt’s experiments involved presenting subjects with an auditory and visual stimulus simultaneously, and asking the subject to introspect and describe their subjective experience of the stimulus. Though Wundt’s experiment revealed that people may have difficulty attending to simultaneous multimodal stimuli, the lack of strict operationalization renders the results less reliable and interpretable. Psychology became a proper science when operationalism (then called operationism) was adopted by the behaviourist school. The behaviourists represented a radical departure from other contemporary schools of psychology, as their focus was purely behavioural, and their science would “never use the terms consciousness, mental states, mind, content, introspectively verifiable, imagery, and the like”^[28] as there were no operational definitions of these concepts in the early 20th century. More recent developments in cognitive science have allowed us to begin to operationalize concepts like consciousness, and from this perspective we can understand Wundt’s aforementioned experiment as a protoscientific inquiry into attentional blink.^[29]

Another field that, somewhat more controversially, fails to meet these standards is modern quantum physics. Much of modern physics is purely mathematical, and posits the existence of various mathematical constructs as real entities. Insofar as entities are described to be equivalent to their mathematical description, physics operates within the realm of mathematical idealism rather than scientific realism.^[30] Describing particles like photons as ‘point particles’ with no radius is a common example of this; a point is a mathematical abstraction, not a physical entity. Virtual particles that mediate interactions in quantum physics are another example. Theories employing such virtual particles may match experimental data (except when they don’t)^[31], but since the virtual particles described in the equations are not real, the equations cannot accurately map onto reality. Such virtual particles and interactions can then be used as a fudge factor to align equations with experimental data, but this is at best instrumentalism, a non-realist philosophy that holds that scientific ideas and equations are useful, but remains ephectic on whether they actually reveal the underlying nature of reality. The prominent physicist Max Tegmark has taken this mathematical idealism to its logical conclusion in his Mathematical Universe Hypothesis, which posits that “all structures that exist mathematically exist also physically”^[32]. Such a field a study cannot be considered a science, as it is not even trying to discover things in the natural world.

Given the stringent requirements entailed by the label “science”, many of the concepts now considered scientific did not begin as scientific ideas. For example, the modern science of chemistry emerged from the pseudoscientific field of alchemy. As our understanding of the scientific method advanced and alchemy adopted more rigorous methods and explanatory models, it was eventually transformed into a true science. On a smaller scale, many scientific ideas emerge through methods of inquiry that bear no inherent relation to science, such as intuition, dreams, or revelation while under the influence of psychedelic drugs. Kekulé's discovery of the ring structure of benzene in a dream is a classic example. Such an idea is not scientific at its outset, but may become so once it has survived falsification and been shown to meet the requirements for rigorous science. This is also true of techniques like acupuncture, which is often explained through Chinese mysticism rather than science, as the knowledge of physiology and electromagnetism required to explain it scientifically did not exist for the majority of its history. It is also possible for fields that were once rigorously scientific to degenerate and lose their rigour; this degeneration will be looked at more closely in the essay on scientific theories.

The fact that scientific discovery can arise from non-scientific sources has been used to argue, most notably by Feyerabend, that there is no scientific method. In his 1975 book Against Method, Feyerabend argues that any definition one puts forth for the scientific method would end up excluding some historical episode of accepted scientific discovery. However, this objection is only sustainable if one denies, as Feyerabend did, the distinction between the context of discovery and the context of justification. There need be no strict rules by which scientific theories are discovered, however there should be a logic by which their validity is determined.⁸ That the human factor results in these two contexts being insufficiently segregated in practice does not mean that no scientific method exists: rather it means that scientists frequently fail to live up to the exacting standards it demands.

The Limits of Science

While it is important to apply scientific rigour wherever possible, it is equally important to recognize the situations where science cannot be applied. Failing to understand the inherent limitations of science results in a naive scientism that is as detrimental to the search for knowledge as cutting out a sheep's liver to predict when next it will rain. This essay may be seen as promoting a form of scientism, so a disambiguation of this term is required. 'Scientism' is first attested in the late 19th century, where it was used as a pejorative attacking science that was unmoored from and dismissive of Christian metaphysics⁹ : "the anti-metaphysical temper of nineteenth century civilization" to quote Shaw.^[33] The term was popularized in the 1940s by Hayek in response to the endeavour to import the methods of physics and chemistry into the social sciences, and remove the "human factor". Today scientism in its most extreme form argues that all things can or will be known through the application of the scientific method, that only positivistic knowledge claims are meaningful (those claims that can be verified mathematically or empirically), and that therefore science is the only worthwhile intellectual pursuit. A weaker form asserts that science is the most reliable form of knowledge within its range of applicability, but that there are meaningful questions it is not capable of answering. The tools of science should therefore be used as widely as possible, with a sophisticated understanding of the conditions under which they can be applied. I take the latter position.

It is important then to understand the limits of science. The most well-known is that science cannot be used to determine appropriate value judgements, or what actions should be taken in light of material conditions (Hume’s famous is-ought problem). Science is therefore incapable of deciding between, for instance, Mill's utilitarianism and Kant's deontological ethics. Arguments over such preferences or values must remain within the realm of philosophy. Philosophy can also develop metaphysical axioms like naturalism or realism that can then be adopted, as they are in science. Science cannot be used to defend naturalism, as a system cannot be used to defend its own axioms - even pointing to the successes of science as a defense of naturalism implies a preexisting system of values that prizes such success. We can then demarcate the domains of science and philosophy: science can be used to make statements about the natural world but cannot say anything about values, while philosophy can be used to argue values but cannot make reliable statements about how the world works.

Apart from philosophy there is also qualitative research and history. Qualitative data is undeniably useful - most of the decisions we make every day are based on qualitative rather than quantitative considerations - however it lacks the reliability and observer invariance that is required in science. Qualitative research is less systematized than quantitative scientific research, which means that historically research into a domain has often begun at a qualitative level to develop hypotheses before quantitative tools are developed to rigorously test them. For example, a qualitative study assessing the personality or intelligence of individuals via unstructured interviews would likely correlate with the results obtained from quantitative tests^[34], but would be far less reliable. Fields of social science like sociology and anthropology employ both quantitative and qualitative methods to build their knowledge base, and while both methods contribute to our understanding, only the quantitative¹⁰ methods have the potential to meet the standards of reliability required by science.

From Qualitative Research: Defining and Designing (2013)

History also cannot be a field of science, for the reason that "non-reproducible single occurrences are of no significance to science."^[35] Science builds a corpus of results that are independently reproducible, a methodology which is impossible to apply to the study of history. In science the amount of available data grows as new techniques are discovered and experiments are carried out. In history there is a monotonic decline in the total amount data over time - even when new historical data is made available through discoveries of ancient texts or ruins, this merely represents existing data that has been made more accessible; there is no way to create new historical artifacts. This is not to say that science cannot be applied to understand history: history should be fully consilient with the results of scientific investigation, and should adopt the metaphysical basis of science where possible to increase reliability. However, a statement about the past can only be considered scientific if it permits falsification through novel observations of the present. For instance, the theory that man evolved from apes could in principle be falsified by novel genetic evidence. In other cases, it may be possible to use scientific reasoning to develop a model that could potentially explain a historical event, and yet not be possible to decide between competing models. This is because theories are underdetermined by evidence (multiple theories could potentially explain the same data), and it is not possible to run the experiments required to choose between models.

The Value of Science

Despite the manifest power of the scientific method, it cannot be held that science or the advancement of scientific knowledge is unequivocally good. Science has the potential to alter society and indeed the world in ways that are not entirely foreseeable, and not obviously for the better. Science¹¹ has provided humanity the tools of its own destruction, with innovations enabling the production of numerous ills: physical and psychological weapons; superstimuli like fast food, pornography, and social media; and the means to radically alter the natural environment in destructive ways. With the current pace of technological innovation it is incredibly difficult to predict what the world will be like in 50 years, or even what humans will be like, what with ongoing research in genetic engineering and the burgeoning philosophy of transhumanism. Nor is this disruption a new phenomenon: it has been hundreds of years since since science upended humanity's self-image as a divine creation, situated in the center of the universe, upon an unmoving Earth.

Many philosophers have criticized the scientific worldview (or the commitment to reason that it requires) for its effects on culture. Nietzsche considered that Socrates, as the avatar of pure reason, represented a degeneration in Athenian culture, privileging rationality over instincts and aesthetic values.¹² Oswald Spengler, in The Decline of the West, similarly describes how the emergence of a rational and scientific worldview transforms a culture – something organic and connected to nature – into a civilization – a mechanical and intellectual construct: “Scientific worlds are superficial worlds, practical, soulless and purely extensive worlds. ... The brain rules, because the soul abdicates.”^[36] Heidegger, one of the founding figures in the philosophical school of phenomenology, was concerned that the scientific practice of reducing things to data points and calculations had resulted in a society that valued efficiency and productivity over meaning and purpose. Both Spengler and Heidegger were also concerned about the conquering aspect of science – the drive to understand, exploit, and subjugate nature to our own ends – and the effect this technological means-to-an-end worldview would have on individuals and society at large.

While these critiques of science are important, they are value judgments, and hence, as discussed previously, not decidable through the use of science itself. The role of science is not to opine on whether science should or should not be pursued, but to produce statements that are true regardless of one’s opinions or perspectives. This presentation of the scientific method is, however, itself idealist. That these philosophical positions underlie science when it is done properly does not mean that they are actually always followed in the real world - something that will become abundantly clear when investigating the human side of science. That science is often flawed may be an indictment of science, however it is not thereby a promotion of other methods of inquiry. Science, on the occasions where it is done properly, is still the most reliable means we have of acquiring knowledge about the world. The foremost moral obligation of anyone claiming to practice science must then be to do it properly.

---

References:

1. Brush, S. G. (1974). Should the History of Science Be Rated X? Science, 183(4130), 1164–1172. https://doi.org/10.1126/science.183.4130.1164

2. Feyerabend, P. (2020). Against method: Outline of an anarchistic theory of knowledge. Verso Books.

3. Westfall, R. S. (1973). Newton and the Fudge Factor. Science, 179(4075), 751–758.

4. Fisher, R. A. (1936). Has Mendel’s work been rediscovered? Annals of Science, 1(2), 115–137. https://doi.org/10.1080/00033793600200111 "the data of most, if not all, of the experiments have been falsified so as to agree closely with Mendel's expectations."

5. Impey, C. (2013). Science Literacy of Undergraduates in the United States. Organizations, People and Strategies in Astronomy Vol. 2, 353–364.

6. Science and Technology: Public Attitudes, Knowledge, and Interest | NSF - National Science Foundation. (n.d.). Retrieved July 8, 2023, from https://ncses.nsf.gov/pubs/nsb20207/public-familiarity-with-s-t-facts

7. Siegel, E. (2020, July 30). You Must Not ‘Do Your Own Research’ When It Comes To Science. Forbes. https://www.forbes.com/sites/startswithabang/2020/07/30/you-must-not-do-your-own-research-when-it-comes-to-science/

8. Bekelman, J. E., Li, Y., & Gross, C. P. (2003). Scope and impact of financial conflicts of interest in biomedical research: A systematic review. JAMA, 289(4), 454–465. https://doi.org/10.1001/jama.289.4.454

9. MacCoun, R. J. (1998). Biases in the interpretation and the use of research results. Annual Review of Psychology, 49, 259–287. https://doi.org/10.1146/annurev.psych.49.1.259

10. Gauchat, G. (2012). Politicization of Science in the Public Sphere: A Study of Public Trust in the United States, 1974 to 2010. American Sociological Review, 77(2), 167–187. https://doi.org/10.1177/0003122412438225

11. Trust in science is becoming more polarized, survey finds. University of Chicago News. (2022, January 28). https://news.uchicago.edu/story/trust-science-becoming-more-polarized-survey-finds

12. Langbert, M., Quain, A. J., & Klein, D. B. (2016). Faculty voter registration in economics, history, journalism, law, and psychology. Econ Journal Watch, 13(3), 422-451.

13. Sanilac, J. (2022, December 12). Trust Networks: How We Actually Know Things https://www.jsanilac.com/trust/

14. Green, C. D. (1992). Of Immortal Mythological Beasts: Operationism in Psychology. Theory & Psychology, 2(3), 291–320. https://doi.org/10.1177/0959354392023003

15. Gauch, H. G. (2003). Scientific method in practice. Cambridge University Press. p. 106. https://books.google.com/books?id=iVkugqNG9dAC

16. Alhazen & Smith, A. Mark (2001), Alhacen's Theory of Visual Perception: A Critical Edition, with English Translation and Commentary of the First Three Books of Alhacen's De Aspectibus, the Medieval Latin Version of Ibn al-Haytham's Kitab al-Manazir, DIANE Publishing, p. 372 & 408, ISBN 0-87169-914-1

17. Bacon, F. (1620/2014). Novum Organum: Or True Suggestions for the Interpretation of Nature. Edited by Joseph Devey. Project Gutenberg. p. 25 https://www.gutenberg.org/files/45988/45988-h/45988-h.htm

18. Galilei, G. (1623/1960). The Assayer: Translated from the Italian by Stillman Drake. In S. Drake (Ed.), The Controversy on the Comets of 1618: Galileo Galilei, Horatio Grassi, Mario Guiducci, Johann Kepler (pp. 151-336). Philadelphia: University of Pennsylvania Press. https://doi.org/10.9783/9781512801453-006

19. Westfall, R. S. (1977). The Construction of Modern Science: Mechanisms and Mechanics. United Kingdom: Cambridge University Press.

20. Newton, I. (1706/2010). Opticks: or, a Treatise of the Reflections, Refractions, Inflections, and Colours of Light (4th ed.). Project Gutenberg. p. 401 https://www.gutenberg.org/files/33504/33504-h/33504-h.htm

21. Kant, I. (1781/1999). Critique of Pure Reason (A. W. Wood & P. Guyer, Trans.). Cambridge University Press.

22. Newton, I. (1687/1846). The Mathematical Principles of Natural Philosophy (A. Motte, Trans.; N. W. Chittenden, Ed.). New York: Daniel Adee. p. 77. Retrieved from https://en.wikisource.org/wiki/TheMathematical_Principles_of_Natural_Philosophy(1846)

23. Dyson, F. (2004). A meeting with Enrico Fermi. Nature, 427(6972), Article 6972. https://doi.org/10.1038/427297a

24. Newton, I. (1678/9). Letter from Newton to Robert Boyle, dated 28 February 1678/9. Cambridge University Library, Cambridge, UK. The Newton Project. Retrieved from https://www.newtonproject.ox.ac.uk/view/texts/normalized/NATP00275

25. Wolfram, S. (2018). A New Kind of Science. WOLFRAM MEDIA Incorporated. p. 737-750 https://books.google.com/books?id=W0GjvgEACAAJ

26. Doolittle, C. (2019, January 12). Epistemology: Testimonialism. In Overview: Core Concepts. The Natural Law Institute. Retrieved from https://naturallawinstitute.com/answers/overview-concepts/

27. Popper, K. R. (1968). Conjectures and Refutations: The Growth of Scientific Knowledge. Harper & Row.

28. Watson, J. (1913). Classics in the History of Psychology—Psychology as the Behaviorist Views it. https://psychclassics.yorku.ca/Watson/views.htm

29. Soto-Faraco, S., & Spence, C. (2002). Modality-specific auditory and visual temporal processing deficits. The Quarterly Journal of Experimental Psychology Section A, 55(1), 23–40. https://doi.org/10.1080/02724980143000136

30. Mathis, M. (2005, July 8). Quantum Mechanics and Idealism. http://milesmathis.com/quant.html

31. Moskowitz, C. (2021, February 1). The Cosmological Constant Is Physics' Most Embarrassing Problem. Scientific American. https://www.scientificamerican.com/article/the-cosmological-constant-is-physics-most-embarrassing-problem/

32. Tegmark, M. (1998). Is "the theory of everything’’ merely the ultimate ensemble theory? Annals of Physics, 270(1), 1–51. https://doi.org/10.1006/aphy.1998.5855

33. Shaw, G. B. (1924). Saint Joan; A Chronicle Play In Six Scenes And An Epilogue. Project Gutenberg Australia. http://gutenberg.net.au/ebooks02/0200811h.html

34. Borkenau, P., Mauer, N., Riemann, R., Spinath, F. M., & Angleitner, A. (2004). Thin Slices of Behavior as Cues of Personality and Intelligence. Journal of Personality and Social Psychology, 86(4), 599–614. https://doi.org/10.1037/0022-3514.86.4.599

35. Popper, K. R. (2005). The Logic of Scientific Discovery (2nd ed.). Routledge. p. 66 http://philotextes.info/spip/IMG/pdf/popper-logic-scientific-discovery.pdf

36. Spengler, O. (1926). The Decline of the West: Form and Actuality. United States: Alfred A. Knopf. p. 353

37. Gormally, C., Brickman, P., & Lutz, M. (2012). Developing a Test of Scientific Literacy Skills (TOSLS): Measuring Undergraduates’ Evaluation of Scientific Information and Arguments. CBE—Life Sciences Education, 11(4), 364–377. https://doi.org/10.1187/cbe.12-03-0026

38. Rancourt, D. (2011, January 5). Activist Teacher: On the False Science of a Fundamental Basis for Progress. Activist Teacher. http://activistteacher.blogspot.com/2011/01/on-false-science-of-fundamental-basis.html

1. Better assessments of scientific literacy do exist, like Gormally et al., 2012,^[37] however they are not in widespread use. The authors of that study also point out that when professors are asked about scientific literacy, they tend to emphasize knowledge of facts: "More than 65.8% of faculty surveyed agreed that all nine skills were “important” to “very important” to scientific literacy. Similarly, most faculty reported that they teach and assess these skills. However, when asked in an earlier open-ended question to state the three most important skills students need to develop for scientific literacy, many responses were related to biology content knowledge, rather than skills."

2. Aristotle was concerned with four causes of objects/phenomena: the material cause, or physical composition; the formal cause, or structure; the efficient cause, or agent that brings the phenomena about; and the final cause, or ultimate purpose of the phenomena.

3. Bridgman originally said “we mean by any concept nothing more than a set of operations; the concept is synonymous with the corresponding set of operations”. By this he meant that concepts have no reality at all apart from their measurement, and that in fact different measures of the same thing (for instance measuring length with a ruler, by trigonometry, or using radar) are all therefore different concepts, which cannot be proven to be identical. I think this approach takes it too far – we may say that there is a metaphysical concept of length, but to talk about it scientifically it must be operationalized.

4. Einstein called relativity a principle theory rather than a constructive theory; constructive theories would match my definition of mechanistic. This will be discussed more in the essay on scientific theories.

5. Abiogenesis as it would be called today is not inherently unscientific, but it must be accounted for by a mechanistic theory. Modern theories involve processes of chemical evolution giving rise to complex molecules and eventually life, however no comprehensive theory is yet known and this is an open research question.

6. "When I meet God, I am going to ask him two questions: Why relativity? And why turbulence? I really believe he will have an answer for the first." - Werner Heisenberg on his deathbed (apocryphal)

7. It is also important to note that there is no universal way to know a priori which natural systems are reducible and which aren't. Imagine the loss to science if physicists had not been able to predict the positions of the planets and we instead offloaded that task onto deep learning models.

8. This also serves to resolve any philosophical disputes about the possibility of induction, as induction is applied in the context of discovery rather than the context of justification.

9. "This is indeed a theory, but a theory of Divine Truth. Scientism endeavours to arrive at pure facts making use of no such theory. … The Scientisms which have now appeared are full of denials, and without such denials would collapse. They deny God as a present Creator; they deny creation in favour of “eternal” alteration. They deny religion and conscience and all spiritual faculties, and all morals except as an expediency."" (*The Greater Origins and Issues of Life and Death*, James John Garth Wilkinson, 1885, p. 29-30)

10. Really it is not the quantitative aspect per se, it is the operationalization; qualitative methods do not have strict operationalizations in the same way that quantitative methods do, although sociologists still use the term to describe their research methodologies.

11. Denis Rancourt argues that science (meaning scientific theory) is not required for significant technological discoveries.^[38] He points to several cases where innovations were in fact considered impossible under the prevailing scientific theory of the time. I would agree that in general theory is not required for technological innovation, though there are some fields like molecular biology where I think theory is essential for interpreting empirical results and making progress. That said, technological advancement still requires the collection of empirical data through the application of the scientific method, and advancement in science and technology is highly correlated.

12. cf. Twilight of the Idols, Judith Norman translation: “The most glaring daylight, rationality at any cost, opposed to instinct, was itself just a sickness, another sickness – and in no way a return to ‘virtue’, to ‘health’, to happiness . . . To have to fight the instincts – that is the formula for decadence: as long as life is ascending, happiness is equal to instinct.”