Are Computer Programs Scientific Theories?

In response to the above quote from a former graduate student, some might say that her algorithm is her theory.

The focus of this document is the question: Are computer programs scientific theories?

Let's begin with some distinctions:

1. Simulation vs. Emulation

   FollowingRoth 1983, let's say that x simulates y means that x is a model of some real or imagined system y,
   and we experiment with x in order to understand y.

   • Typically, x might be a computer program, and y might be some real-world situation.
   • In an extreme case, x simulates y iff x and y have the same input-output (I/O) behavior.

   And, following Habib 1983, let's say that x emulates y means that either:

   1. computer system x interprets and executes computer system y's instruction set by implementing y's operation codes in x's hardware
      —i.e., hardware y is implemented as a virtual machine on x—
      or
   2. software feature x simulates(!) hardware feature y, doing what y does "exactly" as y does it.
   • Roughly, x emulates y iff x and y not only have the same I/O behavior, but also use the same algorithms and data structures.

   This suggests that there is a continuum or spectrum,
   with "pure" simulation at one end (I/O-equivalent behavior),
   and "pure" emulation at the other end (behavior that is equivalent with respect to I/O, all algorithms in full detail, and all data structures).

   So, perhaps there is no real distinction between simulation and emulation except for the degree of faithfulness to what is being simulated or emulated.

   ---

2. Simulations vs. "The Real Thing"

   The term "simulation" has a sense of "imitation" or "unreal":

   • A simulation of a hurricane is not a real hurricane.
   • A simulation of digestion is not real digestion.

   But there are cases where a simulation is the real thing.
   (Question to think about: Would such simulations be better called "emulations"?)

   For example:

   • a scale model of a scale model of the Statue of Liberty is a scale model of the Statue of Liberty
   • a Xerox copy of a document is that document, at least for purposes of using it
   • a PDF version of a document is that document
   • some philosophers and computational cognitive scientists have argued that a computational simulation of cognition is cognition.

   In general, it seems that a simulation of information is that information.

   ---

3. Scientific Theories

   There are at least two views within the philosophy of science about what scientific theories are.

   On the syntactic approach to theories (due to the "Logical Positivists"),
   theories are descriptions of real-world situations expressed in a formal language with an axiomatic structure,
   i.e., a formal system.

   • That is, they are sets of sentences that describe a real-world situation or that codify laws about a real-world situation
     (this is the main sense in which the theory of evolution is a "theory").

   • Theories, in this sense, are semantically interpreted by rules linking them to "observable" phenomena.
     (These phenomena either are directly observable—either by unaided vision or with the help of microscopes and telescopes—
     or are theoretical terms (e.g., "electron") that are definable in terms of directly observable phenomena (e.g., by a vapor trail in a cloud chamber).

   On the semantic approach to theories (due largely to the philosopher Patrick Suppes),
   theories are the set-theoretic models of an axiomatic formal system.

   • Such models are isomorphic to the real-world situation being modeled.
   • (Weaker semantic views of theories see them as "state spaces" or "prototypes" (which are merely "similar" to the real-world situation).)

   ---

4. Simon & Newell 1962; Johnson-Laird 1981, 1988; and Pylyshyn 1984 have all claimed that computer programs are theories,
   in the sense that they are languages for theories or ways to express theories:
   • "1. Computers are quite general symbol-manipulating devices that can be programmed to perform nonnumerical as well as numerical symbol manipulation.
     "2. Computer programs can be written that use nonnumerical symbol manipulating processes to perform tasks which, in humans, require thinking and learning.
     "3. These programs can be regarded as theories, in a completely literal sense, of the corresponding human processes. These theories are testable in a number of ways: among them, by comparing the symbolic behavior of a computer so programmed with the symbolic behavior of a human subject when both are performing the same problem-solving or thinking tasks." (Simon & Newell 1962: 97)
   • "Computer programming is too useful to cognitive science to be left solely in the hands of the artificial intelligenzia [sic]. There is a well established list of advantages that programs bring to a theorist: they concentrate the mind marvelously; they transform mysticism into information processing, forcing the theorist to make intuitions explicit and to translate vague terminology into concrete proposals; they provide a secure test of the consistency of a theory and thereby allow complicated interactive components to be safely assembled; they are "working models" whose behavior can be directly compared with human performance. Yet, many research workers look on the idea of developing their theories in the form of computer programs with considerable suspicion. The reason...[i]n part...derives from the fact that any large-scale program intended to model cognition inevitably incorporates components that lack psychological plausibility.... The remedy...is not to abandon computer programs, but to make a clear distinction between a program and the theory that it is intended to model. For a cognitive scientist, the single most important virtue of programming should come...from the business of developing [the program]. Indeed, the aim should be neither to simulate human behavior...nor to exercise artificial intelligence, but to force the theorist to think again." (Johnson-Laird 1981: 185-186.)
   • "[T]he…requirement—that we be able to implement [a cognitive] process in terms of an actual, running program that exhibits tokens of the behaviors in question, under the appropriate circumstances—has far-reaching consequences. One of the clearest advantages of expressing a cognitive-process model in the form of a computer program is, it provides a remarkable intellectual prosthetic for dealing with complexity and for exploring both the entailments of a large set of proposed principles and their interactions." (Pylyshyn 1984: 76.)
   • "[T]heories of mind should be expressed in a form that can be modelled in a computer program. A theory may fail to satisfy this criterion for several reasons: it may be radically incomplete; it may rely on a process that is not computable; it may be inconsistent, incoherent, or, like a mystical doctrine, take so much for granted that it is understood only by it adherents. These flaws are not always so obvious. Students of the mind do not always know that they do not know what they are talking about. The surest way to find out is to try to devise a computer program that models the theory." (Johnson-Laird 1988: 52.)

   The basic idea is that a theory must be expressed in some language.
   If you don't express it in a language, how do you know what it is?
   As E.M. Forster is alleged to have said, "How can I know what I think till I see what I say?".
   And if you don't write your theory down in some language, no one can evaluate it.

   Scientific theories, on this view, are sets of sentences. (I say more about this elsewhere.)

   This is the sense of (scientific) "theory" that is used when people talk about the "theory" of evolution.
   The ordinary, everyday sense of "theory"—in which a theory is a mere conjecture—
   is not what is intended in the phrase "theory of evolution"
   (contrary to what some creationists or intelligent-design proponents might say),
   nor is it what we're talking about when we say that a computer program might be a "theory".

   The sentences have to be in some language:

   • Some theories are expressed in English,
   • some in the language of mathematics,
   • some in the language of formal logic,
   • some in the language of statistics and probability.

   The claim here is that some theories can be expressed in a programming language.

   One advantage of expressing a theory as a computer program is that all details must be filled in.
   That is, a computer program must be a full "implementation" of the theory.

   Of course, there will be implementation-dependent details.

   • For example, if the theory is expressed in Java, there will be details of Java that are irrelevant to the theory itself.
   • So, one must try to ensure that such details are indeed irrelevant.
   • One way to do that is to make sure that two computer programs expressing the same theory in two different programming languages
     —with different implementation-dependent details—
     have the same I/O, algorithmic, and data-structure behavior.

   Another advantage of expressing a theory as a computer program is that you can run the program to see how it behaves and what predictions it makes.
   So, in a sense, the theory becomes its own model and can be used to test itself.

   ---

5. The notion of model is associated with a puzzle called the Puzzle of the Model in the Middle, or the Model Muddle (Wartofsky 1966, 1979; Rapaport 1995).
   There are different uses of the term "model".
   • It can be used to refer to a syntactic domain, as in the phrase "mathematical model" of a real-world situation.
   • And it can be used to refer to a semantic domain, as in the phrase "set-theoretic model" of a mathematical theory.

   This dual, or Janus-faced, nature of models leads to what Smith 1987 calls a "correspondence continuum":

   1. A scientist typically begins with data that he or she then interprets or models using a formal theory;
      so the data are the syntactic domain, and the formal theory is its semantic domain.

   2. The formal theory can then be modeled set-theoretically or mathematically;
      so the formal theory is now the syntactic domain, and the set-theoretic or mathematical model is the semantic domain.

   3. But that set-theoretic or mathematical model can be interpreted by some real-world phenomenon;
      so the model is now the syntactic domain, and the real-world is the semantic domain.

   4. To close the circle: What is that real-world phenomenon but just the kind of data we started with?

   ---

6. However, Moor 1978 (§4) and Thagard 1984 argue that computer programs are not theories,
   on the grounds that they are neither sets of (declarative) sentences nor set-theoretic models of axiom systems.

   In reading Moor or Thagard, here are some questions to ask yourself:

   1. Must a syntactic theory be expressed in declarative sentences?
      (A declarative sentence is a sentence that is either true or else false.)

   2. Must a computer program be expressed in imperative language?
      (An imperative sentence is a sentence that says "Do this!"; it has no truth-value.)

   3. Must a semantic theory be a set-theoretic model of a real-world situation?

   4. Could a computer process—i.e., a program being executed—be a model of a real-world situation?

   ---

References

1. Habib, S. (1983), "Emulation", in Anthony Ralston & Edwin D. Reilly, Jr. (eds.), Encyclopedia of Computer Science and Engineering, 2nd edition (New York: Van Nostrand Reinhold): 602–603.
2. Johnson-Laird, Philip N. (1981), "Mental Models in Cognitive Science", in Donald A. Norman (ed.), Perspectives on Cognitive Science (Norwood, NJ: Ablex), Ch. 7 (pp. 147-191).
3. Johnson-Laird, Philip N. (1988), The Computer and the Mind: An Introduction to Cognitive Science (Cambridge, MA: Harvard University Press), Ch. 3 ("Computability and Mental Processes"), pp. 37-53.
4. Moor, James H. (1978), "Three Myths of Computer Science" British Journal for the Philosophy of Science 29(3) (September): 213-222.
5. Pylyshyn, Zenon W. (1984), Computation and Cognition: Toward a Foundation for Cognitive Science (Cambridge, MA: MIT Press), Ch. 3 ("The Relevance of Computation"), pp. 48-86, esp. the section "The Role of Computer Implementation" (pp. 74-78).
6. Rapaport, William J. (1995), "Understanding Understanding: Syntactic Semantics and Computational Cognition", in James E. Tomberlin (ed.), AI, Connectionism, and Philosophical Psychology, Philosophical Perspectives, Vol. 9 (Atascadero, CA: Ridgeview): 49-88; reprinted in Toribio, Josefa, & Clark, Andy (eds.) (1998), Language and Meaning in Cognitive Science: Cognitive Issues and Semantic Theory, Artificial Intelligence and Cognitive Science, Vol. 4: Conceptual Issues (New York: Garland): 73-88.
7. Roth, P.F. (1983), "Simulation", in Anthony Ralston & Edwin D. Reilly, Jr. (eds.), Encyclopedia of Computer Science and Engineering, 2nd edition (New York: Van Nostrand Reinhold): 1327–1341.
8. Simon, Herbert A., & Newell, Allen (1962), "Simulation of Human Thinking", in Martin Greenberger (ed.), Computers and the World of the Future (Cambridge, MA: MIT Press): 94-114
9. Smith, Brian Cantwell (1987), "The Correspondence Continuum", Report No. CSLI-87-71 (Stanford, CA: Center for the Study of Language and Information).
10. Thagard, Paul (1984), "Computer Programs as Psychological Theories", in O. Neumaier (ed.), Mind, Language, and Society (Vienna: Conceptus-Studien): 77-84.
11. Wartofsky, Marx W. (1966), "The Model Muddle: Proposals for an Immodest Realism", in Marx W. Wartofsky, Models: Representation and the Scientific Understanding (Dordrecht, Holland: D. Reidel, 1979): 1–11.
12. Wartofsky, Marx W. (1979), "Introduction", in Marx W. Wartofsky, Models: Representation and the Scientific Understanding (Dordrecht, Holland: D. Reidel, 1979): xiii–xxvi.